Understanding the Prediction Gap in Multi-hop Localization. Cameron Dean Whitehouse

Transcription

1 Understanding the Prediction Gap in Multi-hop Localization by Cameron Dean Whitehouse B.A. (Rutgers University) 1999 B.S. (Rutgers University) 1999 M.S. (University of California, Berkeley) 2003 A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Computer Science in the Graduate Division of the University of California, Berkeley Committee in charge: Professor David Culler, Chair Professor Eric Brewer Professor Todd Dawson Fall 2006

2 The dissertation of Cameron Dean Whitehouse is approved: Chair Date Date Date University of California, Berkeley Fall 2006

3 Understanding the Prediction Gap in Multi-hop Localization Copyright 2006 by Cameron Dean Whitehouse

4 1 Abstract Understanding the Prediction Gap in Multi-hop Localization by Cameron Dean Whitehouse Doctor of Philosophy in Computer Science University of California, Berkeley Professor David Culler, Chair Wireless sensor networks consist of many tiny, wireless, battery-powered sensor nodes that enable the collection of sensor data from the physical world. A key requirement to interpreting this data is that we identify the locations of the nodes in space. To this end, many techniques are being developed for ranging-based sensor localization, in which the positions of nodes can be estimated based on range estimates between neighboring nodes. Most work in this area is based on simulation, and only recent applications of rangingbased localization in the physical world have revealed what we call the Prediction Gap: localization error observed in real deployments can be many times worse than the error predicted by simulation. The Prediction Gap is a real barrier to sensor localization because simulation is an essential tool for designing, developing, and evaluating sensor technology and algorithms before they are actually used in costly, large-scale deployments. The goals of this dissertation are 1) to close the Prediction Gap and 2) to identify its causes in sensor localization. We first establish the existence and magnitude of the Prediction Gap by building and deploying a sensor localization system and comparing observed localization error with predictions from the traditional model of ranging. We then develop new non-parametric modeling techniques that can use empirical ranging data to predict localization error in a deployment. We show that our non-parametric models do not cost significantly more than traditional parametric models in terms of data collection or simulation, and solve many of the prediction issues present in existing simulations. In order to identify the causes of the Prediction Gap in sensor localization, we create hybrid models that combine components of our non-parametric models with traditional parametric

5 2 models. By comparing localization error from a hybrid model with a purely parametric model, we isolate the effects of that component of our data. We use this technique to identify the causes of the Prediction Gap for six different localization algorithms from the literature, and conclude by developing a new parametric model that captures the true characteristics of our empirical ranging data. Professor David Culler Dissertation Committee Chair

6 Dedicated to my family i

7 ii Acknowledgments Thanks to all of my collaborators who provided technical, intellectual, practical, and advisorial help with various aspects of this work. These include but are not limited to Alec Woo, Chris Karlof, Fred Jiang, Cory Sharp, Rob Szewczyk, Jason Hill, Scott Klemmer, Sarah Waterson, Gilman Tolle, Jonathan Hui, Phil Buonadonna, Phoebus Chen, Mike Manzo, Matt Welsh, Sam Madden, Prabal Dutta, Joe Polastre, Naveen Sastry, Tye Rattenbury, Kris Pister, Deborah Estrin, Joe Hellerstein, Bhaskar Krishnamachari, Feng Zhao, Carlos Guestrin, David Wagner, Shankar Sastry and, of course, my dissertation committee. Special thanks to Arianna for her constant support. This work was funded in part by the National Defense Science and Engineering Graduate Fellowship, The UC Berkeley Graduate Opportunity Fellowship, the Siebel Fellowship, the DARPA NEST contract F C-1895, and Intel Research at Berkeley.

8 iii Contents List of Figures v 1 Introduction Sensor Field Localization The Prediction Gap Outline of the Solution Background Sensor Field Localization Single-hop Localization Multi-hop Localization Ranging Theory: The Noisy Disk Model Physical Range Sensors Radio Signal Strength Acoustic Time of Flight Interferometric Ranging RF Time of Flight Localization Algorithms Model Verification Simulation-based Studies Ranging Characterization Studies Localization Deployment Studies Establishing the Prediction Gap The Ranging Hardware Noise Filtering Calibration Dealing with Collisions The Localization Algorithm Distributed Programming Implementation and Debugging Deployment Details Comparing Theoretical and Observed Localization Error

9 iv 3.10 The Prediction Gap Established Closing the Prediction Gap Modeling the sensors and environment Parametric Models Non-parametric Models Empirically Profiling the Physical World Traditional Data Collection High-fidelity Data Collection Generality of an Empirical Profile Comparing Non-parametric Predictions and Observed Localization Error Explaining the Prediction Gap The Experimental Setup Identifying Ranging Irregularities Creating Hybrid Models Parameters and Topology Experimental Results Analyzing Each Algorithm Bounding Box DV-Distance MDS-Map GPS-Free Robust Quads MDS-Map(P) Removing the Prediction Gap Existing Models of Irregularity Non-uniformity of Nodes Radio Irregularity Model Gaussian Packet Reception Rate Model Shadowing and Multi-path Towards a New Parametric Model A Geometric Noise Distribution An Exponential Model of Connectivity Verifying the Model Conclusions Advantages of Our Modeling Techniques Parametric vs. Non-parametric Models Extending Analysis to Other Areas Bibliography 124

10 v List of Figures 2.1 Single- and Multi-hop Localization Error Increase over Distance Localization Ontology The Ultrasound Ranging Hardware Raw Time of Flight Readings Averaging Ranging Data The MedianTube Filter The Effect of Filtering The Effect of Calibration Capture Experimental Setup The Prevalence of Capture The Hood Programming Abstraction Stages of Development and Debugging The Final Deployment Localization Error Vectors The Localization Error Gap The Shortest Path Error Gap The Node Degree Gap The Non-parametric Model Traditional Data Collection The Data Collection Topology Profiling Multiple Environments Closing the Localization Error Gap Closing the Shortest Path Error Gap Closing the Node Degree Gap Ranging Irregularities Ordering of Localization Algorithms Experimental Results Causes of the Prediction Gap The Bounding Box Algorithm Shortest Path Errors

11 vi 5.7 The Effect of Density on Shortest Paths Anchor Corrections and Range Irregularities Robust Quads and Stitching Failure Robust Quads Phase Transition Robust Quads Overly Restrictive Radio Transitional Region Non-Uniformity and the Transitional Region Uniformity of Ultrasound Nodes Radio Irregularity Model Sources of Non-isotropic Ultrasound Ultrasound Connectivity Contours Gaussian Packet Reception Rate Model Ultrasound Connectivity Shadowing, Multi-path and Bit Errors Ultrasonic Emanation Pattern Geometric Noise Distribution Connectivity Characteristics of the Geometric Noise Model Power Relationships Complete Parametric Model Top-down Parametric Model Evaluation A Tracking Deployment Profiling the PIR Sensor

12 1 Chapter 1 Introduction Sensor arrays have long been used to measure phenomena that are distributed through space. Perhaps the most common sensor array today is the digital camera: a CCD can consist of hundreds of thousands of light sensors, called pixels, each of which measures the light emanating from a different point in space. The digital camera, however, is an exceptional case; the straight-line propagation of light allows a lens to focus light from multiple different points in space onto a sensor array that is spatially concentrated, often with all the pixels located within one square centimeter. Other stimuli which cannot be focused with a lens, such as temperature, humidity, and pressure, must be measured in place; each sensor must actually be located at the point in space where a measurement is to be taken. This fundamentally changes the computer system needed to operate an array of in-place sensors: when they must be distributed over meters or even kilometers, it is no longer feasible to run a wire to each sensor. Wireless sensor networks enable one to collect data from a spatially distributed array of sensors. Each node in the network consists of a sensor, a microprocessor, a battery and a radio. The microprocessor can sample from the sensor and process the data as needed, as well as communicate with neighboring nodes. In simplest form, all wireless sensor nodes periodically sample from their own sensors and the data is routed through the wireless network to a gateway node, at which point it is permanently archived in a database. This system allows sensors to be distributed through space by replacing power lines with batteries, replacing data lines with a wireless network, and coupling each sensor with a microprocessor for decentralized control. This system architecture enables a plethora of new sensing applications. For example, humidity gradients can be measured in a forest, helping scientists identify the effects of trees on the water cycle [94]. Vibrations of large structures such as buildings, boats, and bridges, can be

13 2 monitored to understand the effects of earthquakes, and to identify points of structural damage [44, 46]. Networks of heat-sensing nodes can be dropped from an airplane around a wildfire to identify areas of rapid movement [20]. Weather sensing nodes can be used for precision agriculture to monitor the light, temperature, and moisture levels at different points in a vineyard [11]. A complete sensor network application requires a number of different system components to operate correctly, including multi-hop routing, time synchronization, localization of the nodes, power management, etc. Furthermore, each specific application carries a different set of constraints and demands on these components. For example, a network used for precision agriculture may require a long network lifetime, but can tolerate high network loss and latency and the nodes can be localized manually at deployment time. On the other hand, a network dropped over a wildfire may only need to function for a few hours but the locations of the nodes must be determined by the nodes themselves. Lastly, a network for structural monitoring may need extremely precise time synchronization to correlate vibrations in space and high sampling rates to capture high-frequency vibrations, straining the network bandwidth. Thus, while the primary goal of a wireless sensor network is simply to replace a wired network, its spatially-distributed and resource-constrained nature makes it difficult to find a one-size-fits-all solution; as one moves along the different axes of the application space, one must manage anew the complex interactions and trade-offs among hardware capabilities, resource constraints, and application requirements. 1.1 Sensor Field Localization In this study, we focus mainly on the task of sensor field localization, which is the process of identifying the locations of the nodes in a wireless sensor network. Location information is critical for essentially all sensor network applications; without knowing the location of a sensor, the data being produced cannot be interpreted. As such, localization is one of the critical system components required by the applications mentioned above. However, it is also an application of its own. In fact, a large fraction of the applications for wireless embedded systems falls under the category of asset or object localization, identification, and tracking. As an application, localization in turn also requires a broad array of other system components and services, sometimes including specialized hardware and drivers, collision detection, multi-hop routing, neighborhood management, modeling and simulation techniques, etc. There are several ways to localize sensor nodes. For example, nodes can be localized at deployment time using a GPS or DGPS receiver that is attached to the person deploying the

14 3 nodes [18]. A survey-grade device can be used to localize the nodes after deployment [28]. A closed-loop system including a pan/tilt laser that is detectable by the nodes can provide similar accuracy without human intervention [32, 75]. Coarse locations can be obtained by simply placing beacon nodes with known positions throughout the deployment area and allowing nodes to estimate their positions based on the beacons within radio range. Each of these localization techniques achieves a different balance of human effort before deployment, node effort after deployment, and localization accuracy. The rest of this study focuses on a particular kind of solution called ranging-based sensor localization. Ranging is the process of estimating the distance between two nodes. Each range estimate is used to constrain the location of one node with respect to the location of a neighboring node. When enough constraints exist within a network of nodes, the locations of all nodes are over-constrained and their relative positions can be solved for analytically. If a small number of anchor nodes with known positions are in the network, the locations of all nodes can be determined within the global coordinate system defined by the anchor nodes. Ranging-based localization systems require no special infrastructure such as a laser system or a dense blanket of beacon nodes and no human effort is required during deployment. Instead, these systems require the design of new range sensors, localization algorithms, and signalprocessing techniques. The rational behind focusing on this particular kind solution is that the cost of solving the ranging-based solution once can be amortized over time, providing both high accuracy and simple operation for many deployments. 1.2 The Prediction Gap Accurate models of the sensors and the environment are extremely important for any sensor network application because they are necessary for designing, simulating, and analyzing the data processing algorithms. Although over 100 algorithms for ranging-based localization have been proposed, however, none of them have addressed the issue of accurate ranging models. Nearly all localization studies are based on a very simple model of ranging called the Noisy Disk, which predicts that a node will obtain a range estimate to all nodes closer than a maximum range d max and that all range estimates will exhibit zero-mean Gaussian noise N(0, σ). However, this is often not the case. As a result, the localization error of an algorithm can be several times worse in a real deployment than predicted by simulation; one empirical deployment from the literature documents errors that were up to 8 times worse [92]. The difference between real-world localization error and

15 4 that predicted by simulation is what we call the Prediction Gap. The Prediction Gap is an important, long-standing problem in the localization literature because real deployments are unpredictable and may not meet application requirements, even if predicted to do so in simulation. Worse, since everything known about the range sensors and environment is built into the model used for simulation, any additional error observed in the real world is unexplained, and therefore difficult to improve upon; since simulation does not necessarily predict reality, any improvements made on the algorithm in simulation will not necessarily translate into improvements in the real world. Similarly, a simulation-based comparison of two algorithms in simulation will not necessarily predict which algorithm will perform better in a real deployment. Without simulation, the user is reduced to development through trial-and-error in the field, which can be especially problematic for mission critical deployments which can only be deployed once, such as forest fire tracking, or for large deployments with 1000 s of nodes where the cost of redeployment is prohibitive. Ultimately, the Prediction Gap is a quantification of the inadequacy of simulation for ranging-based localization today; a large Prediction Gap is an ominous sign in a field where real-world deployments are costly, difficult, and scarce, and most studies are solidly grounded in simulation. The Prediction Gap, however, is not a problem unique to ranging-based localization, which is only one instance of a class of sensor network applications that perform real-time data processing. Unlike the more common class of data collection applications, which collect an opaque data stream for later human processing, data processing applications must actually process the sensor data and infer some physical property about the world, with no human intervention. This means that they must be prepared for noise, failures, and other anomalies in the data because they cannot utilize post-hoc application development, in which the user can repeatedly clean the data and tune the algorithms until the application works. Any application that must operate autonomously with no human intervention and no closed-loop feed-back must be grounded in an accurate model of the sensors and environment. This is acutely evident, for example, in multi-hop routing, for which the community is still improving on models of the radio link layer that took years to develop. It is also evident in applications like tracking, which use passive infrared or magnetometer sensors to infer the locations of moving objects in the world. Finally, it is particularly true for applications where algorithmic properties can change dramatically at scale and/or with different network topologies, and thus where simulation, not deployment, must be the primary mode of development.

16 5 1.3 Outline of the Solution One solution to this problem for ranging-based localization is to create a more accurate model of the environment and range sensor. However, doing so would require that we already understand the causes of the Prediction Gap; in order to build a new model, we must understand which aspects of the environment and range sensor are different from the idealized Noisy Disk model and are causing additional errors in our localization algorithm. It is possible to collect ranging data, analyze it, and postulate a new model, a process we call bottom-up modeling. However, actually validating any new model is a difficult process that requires a real-world localization deployment to be compared with a simulation using the new model, a process we call top-down modeling. This validation process must be repeated whenever the environment, range sensor, or algorithm changes. Instead of trying to create an improved parametric model of ranging, we use non-parametric models which take ranging data collected in the real world and use it directly in simulation. This process avoids the need to reduce the empirical data set to a simple set of parameters and also maintains the integrity of the data set more than a parametric model might, preserving any anomalies and subtle structural artifacts that may be overlooked. In Chapter 4, we validate this modeling process by comparing localization error from a real-world deployment to that predicted by simulation, and find that our non-parametric model predicts the real deployment much more accurately than the Noisy Disk model. Because the non-parametric model does not assume any structure of the data, this validation process does not need to be repeated whenever the deployment scenario changes. Instead, we only need to collect a new set of empirical ranging data with the new environment or range sensor. The main advantage of non-parametric models is that they can be created without requiring the user to model the structure of the data. The corresponding disadvantage is that they do not provide any insight into this structure. Thus, non-parametric models are very useful for creating realistic simulations, but are not very useful for inspiring new algorithmic designs. To remedy this shortcoming, Chapter 5 demonstrates a technique for identifying which components of the data are affecting the localization algorithm. The technique makes use of hybrid models, which use parametric models for some components of the data and non-parametric models for others. If we hypothesize that the Prediction Gap is caused by a particular aspect of the Noisy Disk model, such as the assumption of Gaussian noise, we can create a controlled experiment in which one simulation uses the Noisy Disk model and another simulation uses a hybrid model with the Noisy Disk model of connectivity and a non-parametric model of empirical noise. If the two simulations produce the

17 6 same localization error, we conclude that the Gaussian noise assumption is a sufficient model of empirical noise. On the other hand, if the two produce different localization error, we can conclude that empirical noise is not Gaussian. Thus, hybrid parametric/non-parametric models enable a scientific process through which we can identify the structural components of the data that most affect a particular localization algorithm. After achieving a sufficient understanding of the data set, a new parametric model can be derived. This algebraic form would useful, for example, when deriving properties or performing mathematical proofs about an algorithm. In Chapter 6, we derive a model based on a geometric distribution of noise and a log-normal model of attenuation that includes shadowing and multi-path effects. We validate the model by comparing its predicted localization error to that of an empirical deployment, and find that it predicts better than the Noisy Disk model and as well as the nonparametric model. The Prediction Gap is not a problem specific to ranging-based localization and our solution can be applied to any data processing application. In Chapter 7, we show how these techniques can be extended to model a passive infrared sensor, for example, to simulate and analyze a tracking application.

18 7 Chapter 2 Background Localization is a very broad and active area of research that spans the areas of hardware design, signal processing, and algorithms and has been applied in a diverse set of application domains including sensor networks, ubiquitous computing, military applications, and inventory management. In this chapter, we place our work within the context of this larger body of literature, outlining key concepts and pointing out important previous studies. We first define the problem of localization in Section 2.1 and an ontology of different versions of this problem. We then describe the task of ranging, including a common theoretical model of ranging in Section 2.2 and several existing implementations of range sensors in Section 2.3. In Section 2.4 we provide an overview of six localization algorithms from the literature, and in Section 2.5 we explore the literature for existing evidence or explanations of the Prediction Gap. 2.1 Sensor Field Localization In the problem of sensor field localization, a sensor field is usually defined as n nodes in a two-dimensional plane, although most of the definitions and solutions in this area can be straightforwardly extended to three dimensions. The first m nodes are termed anchor nodes and have known locations in a global coordinate system. Anchor nodes may be localized using Differential GPS (DGPS) [18], surveyor-quality laser range finders [28], tape measures [97], or other means, but due to the challenges of manual localization, the number of manually localized nodes is generally much smaller than the number of nodes in the network: m << n. In ranging-based localization, each node i can obtain a distance estimate ˆd ij to another

19 8 neighbor j that is some function of the true distance d ij between them. ˆd ij = f(d ij ) (2.1) Some range estimates, however, are failures, which means that a pair of nodes does not obtain a valid distance estimate at all. Failures can occur for a variety of reasons, including hardware failure, noise filtering, or a low signal to noise ratio. These failures can be denoted by a null value ˆd ij = ø. The nodes and the distance estimates between them form a graph G = (V, E), where V = n and ø failure between i and j e ij E = (2.2) f(d ij ) otherwise The task of sensor field localization is to derive the positions of the n m unlocalized nodes from the ranging graph G. The general problem of sensor field localization can be divided into four sub-classes. Centralized localization assumes that the entire graph G can be collected to a single location and can be used to localize all nodes. Distributed localization refers to the process in which each node derives its own location using only locally available information, which is typically only a subsection of the graph G. Absolute localization requires all nodes to be localized in a single global coordinate system defined by the anchor nodes. Relative localization requires each node to be localized relative to its neighbors in a locally-defined coordinate system. The benefit of relative localization techniques is that they can be used even when m = 0. If m 3, a local-global coordinate transform can be derived to convert any relative localization algorithm to an absolute localization algorithm Single-hop Localization In Single-hop localization, each node i has a direct range estimate to at least three anchor nodes. For each such anchor node j, this estimate ˆd ij can be used to relate the unknown coordinates (x i,y i ) of the unlocalized node to the known coordinates (x j,y j ) of the anchor node using the standard distance formula: ˆd 2 ij = (x i x j ) 2 + (y i y j ) 2 ˆd 2 ij = x2 i 2x ix j + x 2 j + y2 i 2y iy j + yj 2 (2.3) A system of three or more such equations can be linearized by subtracting one of the equations from the rest to remove the quadratic terms, leaving two variables in two or more linear equations. For example, the following three distance equations

20 9 (a) Single-hop Localization (b) Multi-hop Localization Figure 2.1: Single- and Multi-hop Localization differ in that in a) single-hop localization, all nodes have range estimates to at least three anchor nodes while in b) multi-hop localization, nodes must use range estimates to other unlocalized nodes. Here, black nodes are anchor nodes while white notes need to be localized.

21 10 ˆd 2 ij = x 2 i 2x i x j + x 2 j + y 2 i 2y i y j + y 2 j ˆd 2 ik = x2 i 2x ix k + x 2 k + y2 i 2y iy k + y 2 k ˆd 2 ih = x2 i 2x i x h + x 2 h + y2 i 2y i y h + y 2 h (2.4) can be reduced to the following two equations, which are linear in x i and y i. ˆd 2 ij ˆd 2 ih = x i (2x h 2x j ) y i (2y h 2y j ) + y 2 j y 2 h ˆd 2 ik ˆd 2 ih = x i (2x h 2x k ) y i (2y h 2y k ) + y 2 k y2 h (2.5) These two equations can be solved for the two unknown variables x i and y i directly. This process is known as tri-lateration. If more than two equations remain, the linear system can be solved approximately using least squares in a process often called multi-lateration. All of the values needed to solve this set of equations, i.e. the range estimates ˆd ij and the anchor coordinates (x j,y j ), are immediately available to the node through local radio communication. Therefore, single-hop localization can be trivially executed in a distributed fashion, where each node localizes itself using only locally available information. It can be difficult to apply to sensor field localization, however, because of the high proportion of nodes that must be manually localized in order to fully cover a sparsely connected sensor network with anchors. In a grid-like network where each node is connected to all eight of its immediate grid neighbors, more than one in every four nodes would need to be manually localized for every node in the network to be connected to at least three anchor nodes. An alternative to tri-lateration is called RF profiling, developed for the RADAR localization system at Microsoft [5]. RF profiling requires a pre-deployment stage in which the radio signal strength (RSS) of each beacon is recorded at each position in the two dimensional region to be localized. The readings taken at a particular position can be called the RF profile of that position. At a later time, a node with unknown location matches the RF profile of its current position to the profiles of the positions already recorded. RF profiling is a single-hop technique because it still requires each mobile node to have direct radio connectivity with several anchor nodes. One disadvantage over tri-lateration is that the user must profile the entire two-dimensional region in which localization is to take place. However, this tedious process also allows the technique to deal with environmental sources of systematic error such as walls and furniture that can disturb RF signal propagation. Indeed, this technique was motivated by an initial implementation of RADAR which

22 11 found that RSS was inadequate for distance estimation indoors, even with an attenuation model that accounted for walls and other objects [5]. RF profiling allowed RADAR to achieve approximately 4m localization error indoors using nothing but RSS from base stations. Single-hop localization is well understood and several commercial systems and academic prototypes have been built. GPS is a well known system that uses an expensive infrastructure of highly synchronized satellites and multi-lateration to find the position of mobile nodes on the earth s surface using RF time of flight [36]. Cricket performs multi-lateration indoors using ultrasonic time of flight [70]. Besides RADAR, several systems have employed RF profiling for RSS-based localization using several different types of radios, including [19, 29, 49], VHF [15], cellular radios [84], and most recently low-power wireless sensor networks [57]. A recent study has shown that Bayesian inference can achieve similar results without pre-collecting a complete RF profile [58], although this technique does not remove the requirement of dense anchor nodes Multi-hop Localization In Multi-hop localization, nodes are not directly adjacent to multiple anchor nodes and must use non-adjacent anchors and range estimates for localization, as illustrated in Figure 2.1. This makes the problem fundamentally more difficult than single-hop localization for two reasons. First, Equations 2.4 can no longer be linearized because the remaining quadratic terms in Equations 2.5 such as yj 2 can be eliminated only if j is an anchor node and y j is a constant value. Second, the range estimates and anchor node coordinates required to make Equations 2.5 a fully constrained system of equations are not necessarily available through single-hop communication and must be obtained through distributed routing or dissemination algorithms. Multi-hop localization is therefore neither a simple linear optimization nor is it computable with only local information. One main difference between single- and multi-hop localization is that multi-hop algorithms must be evaluated at scale. In single-hop localization, the accuracy observed in a single-cell deployment can be generalized to larger multi-cell deployments because each cell is roughly independent. In other words, a system shown to work in one room can reasonably be expected to work across an entire building if each room has enough anchors. The performance of a multi-hop algorithm on a sensor network of 10 nodes, however, can be completely different from its performance on 1000 nodes. Furthermore, each algorithm cannot be evaluated on only one topology, but must be evaluated on a broad range of network topologies. For these two reasons, multi-hop localization research is mainly focused on theoretical analysis and simulation with relatively few empirical

23 12 systems and prototypes. 2.2 Ranging Theory: The Noisy Disk Model A ranging model is the function f(d ij ) from Equation 2.1. For theoretical analysis and simulation of multi-hop localization, range estimation is almost universally modeled with the Noisy Disk, which has two components: noise and connectivity. The connectivity component is parameterized by a value d max and states that a node will obtain a range estimate to all nodes within d max and to no nodes beyond d max. The noise component models the differences between the range estimates and the true distances using a Normal distribution with standard deviation parameter σ. In some instances, a variant of this model has used a uniform noise distribution instead of the normal distribution. When using Gaussian noise, the Noisy Disk defines the distance estimate ˆd ij between nodes i and j in terms of the true distance d ij as N(d dˆ ij,σ) d ij d max ij = ø otherwise. The connectivity component of the Noisy Disk model is also known as the Unit Disk model of connectivity. (2.6) Researchers generally acknowledge that noise is not perfectly Gaussian or uniform and that connectivity is not perfectly disk-like. Regardless, the Noisy Disk model is universally considered to be good enough for the simulation and evaluation of multi-hop localization algorithms and is ubiquitous in the sensor localization literature. Theoretical analyses have successfully used the Noisy Disk model to derive the maximum likelihood solution to localization [103], lower bounds on localization error [13, 81], and specific properties about localization algorithms [60]. The Noisy Disk Model is most commonly used to evaluate and compare algorithms in simulation [1, 2, 4, 21, 42, 67, 84, 87]. Several projects have collected empirical ultrasound data [82] or RSS data [66,90] to derive realistic values for the parameters d max and σ, which are then used to simulate the behavior of various localization algorithms. Other studies use these parameters for sensitivity analysis by, for example, measuring localization accuracy while varying d max from 1.1 to 2.2 times the average node spacing and σ from 0 to 50% of d max or similar values [50, 63, 79, 82].

24 Physical Range Sensors Any signal that changes reliably over distance can be used as a range sensor. For example, magnetic fields can be used to localize objects in three dimensions with millimeter accuracy [71], although the limited range of a few feet makes it difficult to apply to sensor field localization. The physics of the sensor determines how closely the resulting range estimates match the Noisy Disk model of ranging. This section describes the physical principles underlying several types of range sensors that are particularly well suited for sensor field localization: they all have relatively long range in roughly all directions and use small, cheap and low-power hardware that require simple signal processing that can be performed on a sensor node. We do not discuss ranging techniques that assume a subset of more powerful nodes, such as laser range finding techniques [32,76,100] or mobile nodes [38, 93] Radio Signal Strength Radio signal strength (RSS) is the power with which a radio signal is received. If the transmission power is known, RSS can be used to estimate distance based on a model of signal attenuation over distance. One such model can be derived from simple principles of physics: assuming an isotropic transmitting antenna and a near-ideal environment, the radio signal should emanate from the transmitter in a sphere. Therefore, signal strength at a receiver with distance r from a transmitter is proportional to Ar A s where A r is the aperture (surface area) of the receiver and A s is the surface area of a sphere with radius r. More precisely, RSS = P ta r (4πr) 2 (2.7) where P t is the transmission power. Since A r is constant and the area of a sphere is proportional to 1 r, RSS will decrease by a factor of every time r doubles, i.e. the received power will decrease by 10log 10 (4) = 6dB as r doubles. The model above assumes a coefficient of attenuation α = 2, based on the rate of growth of the area on a sphere. However, in reality antennas are not isotropic and RF power does not radiate in a sphere. Furthermore, a real deployment environment is never ideal, and so the coefficient of attenuation can be significantly higher than 2. Other more realistic models have been proposed that combine theory with empirical observation, including Nakagami and Rayleigh [72] fading models. Perhaps the most commonly used path loss model is the the log-normal model [72], which postulates

25 Average Signal Strength over Distance Small Error Signal Strength (V) Large Error Distance (ft) Figure 2.2: Error Increase over Distance depends on both noise and attenuation rate. As the attenuation rate flattens out, differences in signal strength become small relative to noise levels. logarithmic attenuation over distance and Gaussian noise, as given by RSS(d) = RSS(d 0 ) + 10α log 10 ( d ) + X σ (2.8) d 0 where d 0 is a reference distance and X σ is a Gaussian random variable. In contrast to the theoretical model, the coefficient of attenuation α in these models is a parameter derived from empirical data. RSS noise is the amount by which RSS can vary at a single distance in a particular environment and, together with the coefficient of attenuation, determines the overall ranging error of RSS. Because RSS attenuates at an ever decreasing rate, the difference in signal strength between 1m and 2m will be equal to or larger than the difference between 10m and 20m. Thus, as distance increases, changes in signal strength due to distance become small relative to noise, even if the level of noise remains the same over distance. A constant level of noise therefore results in ever increasing error when signal strength is used to estimate distance; if RSS noise is sufficient that we cannot tell the difference between 1 and 2m, we also cannot tell the difference between 10m and 20m. This effect is illustrated in Figure 2.2, which shows how noise at 5 10 foot distances translates into small error, while similar noise levels at foot distances translates into large error. In many ways, RSS ranging is the ideal range technology for wireless sensor networks:

26 15 it requires no additional hardware and almost no computational costs. However, even with a single pair of stationary nodes in a stationary environment, RSS is subject to high levels of noise. Furthermore, individual radios can vary significantly in both transmission strength and receptivity, especially in low-power radios [34, 95], and the effect of reflectors and attenuators can dominate the effect of distance on RSS, giving RSS a reputation for being extremely noisy and unsuitable for multi-hop localization [10, 31, 87]. Although projects such as RADAR have used RSS with RF profiling, most empirical studies that use RSS directly for range estimation have yielded inconclusive or negative results, even outdoors. One study explored RSS ranging outdoors in both an open and a heavily wooded environment using two nodes, but only promised a standard deviation of error near 50% of the range at best [90]. RSS ranging was shown to be effective for indoor localization to within 1.8m in another study, but only when the nodes had a 2-3 meter spacing and RSSI was measured using the Berkeley Varitronix Fox receiver, a high-fidelity Wi-Fi propagation analyzer [66]. The low-power radios that are common in sensor networks are even more difficult to use for ranging. Several studies that characterized RSS data using low power radios decided not to use these radios for localization [34, 40] or later rejected RSS in favor of other ranging technologies [83, 98]. Today, for real deployments that require sensor field localization, more costly alternatives to RSS such as acoustic, RF time of flight, or laser are being developed to localize nodes outdoors in open spaces with only 10 or even 2 meter spacing [32, 48, 68, 82, 88]. This reflects a general lack of confidence in RSS ranging in the community, although no conclusive results have shown RSS ranging to be impossible. We demonstrate in a separate study that RSS can indeed be used for multi-hop, sensor field localization and can even achieve results comparable to GPS, achieving near 4m average accuracy on a 49 node 50x50m network [97]. However, this was only possible by using the techniques described in subsequent chapters to create a predictable deployment environment Acoustic Time of Flight The time of flight (TOF) of an acoustic signal is the difference between transmission time t t and receive time t r. TOF can be multiplied by the speed of sound to infer the distance between the transmitter and receiver according to the following equation ˆd ij = (t r t t ) c (2.9) where v is approximately 331.5(0.6 θ)m/sec and θ is the temperature in degrees Celsius. TOF can be measured in two common ways. If the transmitter and receiver are operating in the same time

27 16 base, the transmitter can send an acoustic pulse at a known time and the receiver can simply observe the time at which it is received [27]. In a sensor field, this technique requires time synchronization between nodes, which has been shown to be accurate to within microseconds, although often at a significant cost in bandwidth and/or energy [59]. Alternatively, the transmitter could send an acoustic pulse and a radio message simultaneously [70]. The RF pulse arrives within 10s of nanoseconds and its reception time is a reasonable estimate of t t for short distances, and the receiver can measure the time difference of arrival (TDOA) of the two signals to estimate the true TOF. As a ranging technique, acoustic TOF is generally more robust to environmental influences than RSS because attenuation and reflection of the signal does not affect the TOF of the line of sight signal; it only affects the volume with which it is received when it arrives. The beginning of a weaker signal may be more difficult to detect, however, and weaker signals may therefore have higher error on average. One way to avoid this problem is to modulate the outgoing signal and measure the phase of the received signal, allowing one to infer the arrival time of beginning of the signal even if it is not directly observable. Girod demonstrated that if the signal is not self-correlating, this modulation technique not only provides increased precision upon detection, but also robustness to multi-path reflections and interference from other transmitters, achieving 5 centimeter accuracy in environments as adverse as a forest, at distances of 10s of meters [28]. As with almost any ranging technique, however, acoustic TOF can always yield very high errors when the line of sight signal is blocked but a reflection of the signal is not. Because of its increased robustness, acoustic TOF has been used with more success in both single- and multi-hop localization than RSS. Many early single-hop localization systems such as AT&T s Active Bats [30] and MIT s Cricket [70] used ultrasonic TOF, as do many robotic systems including CMU s Millibots [62] and the popular Pioneer robot series [22]. More recently, UCLA s AHLoS [82] localization system and a similar system by UIUC [48] are using acoustic TOF towards multi-hop, sensor field localization. UCLA s Acoustic ENSBox [28] uses wideband acoustics in the audible frequencies to localize nodes even in the presence of obstacles such as trees Interferometric Ranging Although radio signal strength and ultrasonic ranging are most commonly used and cited, a new ranging technique has recently been proposed and shows promise for wireless sensor networks. In Radio Interferometric Ranging, two non-colocated nodes A and B transmit radio signals at different frequencies f 1 and f 2, such that the difference between them is small f 1 f 2 << f 1,f 2.

28 17 These signals interact to create a low beat frequency which can be detected at receivers B and C. The phase will be different at each receiver, however, and the relative phase offset will be 2π d AD d BD + d BC d AC (mod2π) (2.10) c/f where c is the speed of light and f = (f 1 + f 2 )/2. In other words, the phase difference between the signals received at C and D is a function purely of the four distances among the four nodes. Repeating this process with other sets of four nodes creates an over-constrained system from which the distances between all nodes can be derived. Because the phase at two different nodes must be compared, this system requires precise time synchronization in the network. Also, Equation 2.10 assumes a line of sight signal from both transmitters to both receivers. Any interference from other transmitters or from reflected signals can change the phase and even the beat frequency observed at the receiver and can cause large errors. Even distant objects in the environment can therefore be an obstacle for this technique, since the range of interferometric ranging has been shown to be much larger than even the effective radio range [47]. In theory, errors due to multi-path are detectable because each distance is being measured multiple times; since the multi-path effects will be different for each pair of transmitters and receivers, erroneous range estimates will be inconsistent with other estimates and can be eliminated. This technique for dealing with multi-path errors, however, has not yet been demonstrated. Without multi-path problems, this technique has been shown to produce range errors with a standard deviation of error near 3 centimeters over ranges of up to 160 meters with the Chipcon CC1000 radio, which is used with the mica2 platform. This technique therefore combines the long range obtained with radio signal strength and the high accuracy obtained with ultrasonic ranging without adding significant hardware or computational costs to the sensor nodes RF Time of Flight Another promising technique for sensor networks is RF time of flight. Historically, RF time of flight has been reserved for systems like GPS that can achieve accurate time synchronization between multiple nodes in an open, outdoor environment. Ultra-wideband (UWB) radios have recently been demonstrated to remove both of these requirements to some degree [53]. By using round-trip time, participating nodes do not need to be synchronized in time, and the short duration of a UWB signal allows the line-of-sight signal to be identified from the midst of reflected signals because it is the first signal to arrive.

29 18 More recently, Lanzisera [51] demonstrated TOF on a 2.4GHz radio with requirements compatible with IEEE radios, which are often used in sensor networks. Node A repeatedly modulates a code that is not self-correlated, similar to the acoustic TOF system by Girod [28]. Another node B receives this signal, buffers it, and retransmits what was received back to A. Because A and B are not time synchronized, B will not likely begin receiving at the very beginning of the transmission, but at some arbitrary point during the first cycle of the code. Nonetheless, A can measure the phase offset between the original transmission and the signal received from B, which should be exactly the time of flight of the radio signal. Repeating this process at multiple frequencies and averaging the resulting range estimates can help reduce the impact of systematic errors due to multi-path reflections. A prototype system was able to achieve RMS error between 1 and 3m in environments including hallways and a coal mine. 2.4 Localization Algorithms There are currently a large number of ranging-based localization algorithms in the literature, each of which uses a different heuristic to infer node locations based on range estimates. Some algorithms assume that a network can be decomposed and localized as several sub-networks; other algorithms assume that range estimates can be added together to create longer range estimates; other algorithms assume that multi-dimensional coordinates can be projected onto a two-dimensional space. Each of these approximations greatly simplifies the sensor localization problem. In this section, we provide an overview of six representative algorithms. Most multi-hop localization algorithms, including the six that we discuss, fall withing two main classes of approximations: the shortest-path and the patch and stitch approximations. Shortest path algorithms approximate the distance between two non-adjacent nodes to be the shortest path distance through the ranging graph G. For example, if node i does not have a direct range estimate ˆd ij to node j, it may use a sum of the range estimates through nodes k and m: ˆd ik + ˆd km + ˆd mj. This sum constitutes a multi-hop range estimate that is a weak approximation of ˆd ij. The shortest-path approximation is that the shortest of all such multi-hop range estimates can be considered equal to a true range estimate. Shortest paths distances can be efficiently calculated in the network using a distance vector algorithm similar to those used in routing. All shortest path distances are initialized to infinity: sp ij =, i,j and one node i initiates the algorithm by transmitting a shortest path update message sp ii = 0. Every node j that hears an update message sets its own shortest path distance to i to be

30 19 the minimum sum over all neighbors k of its ranging estimate to k and the current shortest path between k and i. sp ji = min k ˆdjk + sp ki (2.11) Whenever its shortest path to i improves, node j notifies its neighbors with another update message containing the new value sp ji and the algorithm repeats. Each run of the algorithm only calculates shortest path distances between all nodes and i, the node that initiated the algorithm. Thus, if an algorithm requires shortest paths to all anchors, the algorithm must be initiated by each anchor individually. Once the necessary shortest paths have been created, each algorithm uses them in a different way. The Bounding Box algorithm constrains the location of node i to be within ˆd ij of node j s x or y coordinates. This constraint can be represented as a box with j in the center and edge length 2 ˆd ij. If such boxes are formed around multiple anchor nodes, the position of node i is constrained to be within the intersection of these boxes. The DV-Distance algorithm is very similar except that it defines a circle around node j with radius ˆd ij. Instead of constraining the location of node i to be within this circle, DV-Distance constrains the position to be on the circle. The intersection of multiple such circles defines the location node i. The MDS-Map algorithm uses the shortest path distances between all nodes in the network to form a similarity matrix, which indicates how close each node is to every other node. This matrix is then used to compute the positions using Multi-dimensional Scaling (MDS), a statistical technique that embeds a set of data points in a multi-dimensional space. Patch and stitch algorithms divide the network into small patches that are localized individually with respect to a local coordinate system. Typically, the algorithms form a patch around each node i consisting of all neighbors N i V where j N i e ij ø. The nodes in the overlap N ij = N i N j between patches for nodes i and j have two coordinates, one in the coordinate system of i and one in that of j. These coordinates can be used to derive a coordinate transform between the coordinate systems of the two patches, thereby providing the relative locations of i and j. The relative locations of non-neighbor nodes i and j can be calculated by cascading transforms from multiple overlapping patches between i and j, or a global stitching order can be used to localize all nodes within the same coordinate system. Once the patches are defined, each algorithm uses a different technique to localize them. The GPS-free algorithm uses a process called iterative localization as its patch localization algorithm. In this process, three nodes are assigned initial coordinates in an arbitrary coordinate system.

31 20 Figure 2.3: Localization Ontology The six multi-hop algorithms that we implemented and analyzed are shown in terms of the ontology of localization problems provided in Section 2.1. We do not analyze the single-hop algorithms. These three nodes are used to localize a fourth node. The four nodes can be used to localize a fifth, and so on. The Robust Quads algorithm is very similar except that it limits each step in the iterative process to localizing only those nodes with a low probability of localization error. The MDS-Map(P) algorithm finds the shortest paths between all nodes in a patch and uses MDS to localize the entire patch at once. As such, MDS-Map(P) uses both the shortest path and the patch and stitch approximations. MDS-Map(P) specifies a global stitching order with which the coordinate systems of all patches can be transformed into a global coordinate system. The set of stitched patches S is initialized to the largest patch S = argmax i N i. The set of un-stitched patches is set to be all other patches S = V S. At each step, the next patch to be stitched is determined to be the patch in S with the largest overlap with any patch in S argmax i N i N j i S, j S (2.12) All six algorithms are shown in Figure 2.3 in terms of the traditional taxonomy of localization algorithms from Section 2.1. All of the algorithms are multi-hop localization algorithms, meaning that nodes are not assumed to have a range estimate to three or more anchor nodes. Only MDS-Map is a centralized algorithm because it needs all range estimates at once. All other algo-

32 21 rithms are distributed algorithms. Bounding Box and DV-distance are absolute localization algorithms that require at least three anchor nodes in the network and localize all nodes with respect to the coordinate system that they define. All other algorithms localize nodes relative to each other in a unique but arbitrarily defined coordinate system. 2.5 Model Verification In the previous two sections, we described both the standard theoretical model of ranging and the physics of several common ranging techniques, but have not yet established any relationship between them. In this section, we explore previous studies in both ranging and localization for evidence of a verified relationship between the Noisy Disk model and a common ranging technique. We look specifically for two different types of verification: bottom-up and top-down. In bottom-up model verification, a researcher collects empirical ranging data using a range sensor and verifies through inspection that its structure is similar to the hypothesized model. Bottom-up verification can be performed through formal statistical tests. For example, an assumption of Gaussian noise can be tested with the Jarque-Bera test of Normality [6]. Data that would be used for these tests is typically collected from a single transmitter and receiver pair which are placed at multiple different distances. Top-down model verification compares the effects of empirical ranging data on localization to the effects of a theoretical model, i.e. it defines equivalence to be in terms of the particular usage of the data. If the empirical data yields the same localization results as the theoretical model, the model is assumed to be sufficient. Like bottom-up verification, top-down verification can also be performed through formal statistical tests, such as the t-test. This type of verification has the benefit of testing not only whether all of the properties of the Noisy Disk model are exhibited by the empirical ranging data, but also the reverse: whether all of the properties of the empirical data that affect localization error are captured by the Noisy Disk model. As such, top-down model verification is more convincing than bottom-up model verification. However, In multi-hop localization, bottom-up verification is much more common than top-down model verification because researchers are much more likely to characterize a ranging technology than to use it in a large-scale localization deployment. Table 2.1 summarizes selected existing studies along with both their usage of and their verification of the Noisy Disk model. Columns 1-3 indicate whether the study performed a localization simulation, and whether that simulation relied on the Gaussian noise and Unit Disk connectivity

33 22 models. The 4th column indicates whether the study actually collected empirical ranging data. The 6th and 8th columns indicate whether the study used the data to estimate parameters for Gaussian noise or Unit Disk connectivity models, and the 5th and 7th columns indicate whether the study first performed any formal tests to verify the assumptions of these models before estimating their parameters. Finally, columns 9-11 indicate whether the study performed a localization study using real hardware, and whether it compared the results of this study to a simulation as a means of top-down verification of the Gaussian or Unit Disk model. This table shows that almost no studies performed any verification of the Gaussian or Unit Disk models (columns 5, 7, 10, and 11), even though every single study assumed these models to some extent (columns 2, 3, 6, 8). The only study that does not appear to assume the Noisy Disk model is by Simic [91] because it does not evaluate the algorithm that it proposes either in simulation or on real hardware, nor does it collect empirical ranging data. The derivation of the algorithm, however, does assume Unit Disk connectivity. Similarly, only one study by Stoleru [92] performed top-down verification of the Unit Disk model of connectivity, producing a negative result: connectivity was not sufficiently disk-like to produce results in the real world similar to predictions in simulation Simulation-based Studies Even though the Noisy Disk model has not been verified, most localization studies use the Noisy Disk in simulation to evaluate or compare localization algorithms. This is evident from the fact that nearly all localization studies that use simulation (column 1) also use both Gaussian noise (column 2) and Unit Disk connectivity (column 3) in that simulation. Reliance on the Noisy Disk is perhaps most evident in algorithms that explicitly depend on its particular assumptions and parameterization. For example, in 2001 Doherty proposed an analytical solution to localization using semi-definite programming by assuming an upper bound on the distance between two connected nodes, relying on the strict assumption that no node would underestimate the distance to another node [17]. In 2004, Biswas modified this algorithm by also assuming that range estimates do not overestimate the distance between two nodes [9]. Both of these algorithms were evaluated using the connectivity models that they assume. The only four algorithms that do not use Gaussian Noise in their simulations [10,17,31,79] are evaluating algorithms that are based on hop count, which does not have a noise component. Another three algorithms indicated with footnotes in Table 2.1 do not use Gaussian noise but also

34 Study Name Bounding Box [91] Convex [17] Hop-terrain [79] MDS-Map [87] MDS-Map(P) [86] GPS-free [12] TPS [1] Fading [7] Bits [82] Semidefinite [9] Anisotropic [56] Comparison [50] APS [63] 1 Anchor-free [69] 1 Scaling [41] 1 APIT [31] SpotON [34] Robust [26] RF-tof [52] Acoustic [78] Quantized [67]? Geolocation [8] Millibots [61] Sichitiu [90] Dynamic [83] 2 Context-aware [45] 2 Relative [66] 2 Robust Quads [60] 2 GPS-less [10]? 2? Time & Space [25] 2 Resilient [48] Aensbox [28] Prob Grid [92] Simulates Localization Uses Gaussian Noise Uses Unit Disk Connectivity Collects Empirical Ranging Data Bottom-up Gaussan Verification Estimates Gaussan Parameters Bottom-up Noisy Disk Verification Estimates Noisy Disk Parameters Collects Localization Data Top-down Gaussian Verification Top-down Unit Disk Verification 23 Table 2.1: that perform localization simulations, collect ranging data, or collect localization data. This table indicates whether each study a) assumes and b) validates the Noisy Disk model. For each column, indicates true and indicates false, indicates that the column is not applicable, and? indicates an inconclusive result.

35 24 do not propose a more realistic, validated noise model. Instead, they use uniformly distributed noise. APIT is the only study that does not use the Unit Disk model of connectivity. The algorithm is explicitly designed to handle non-disk like connectivity, and uses an irregular radio connectivity model which later became the basis for the Radio Irregularity Model (RIM) [102]. RIM is a model of radio characteristics that has been derived through a bottom-up verification process. By using the model to evaluate a localization algorithm, APIT is assuming that, because of this bottomup validation, RIM also satisfies the more demanding top-down validation requirements. Because the simulation results were not compared to a real deployment, however, this has not been verified. Several studies that evaluate a localization algorithm with the Noisy Disk model actually derive the model parameters from empirical ranging data. For example, Savvides evaluated the collaborative multilateration algorithm in simulation using parameters derived from ultrasound ranging data [82], Patwari evaluated a maximum likelihood algorithm using parameters derived from RSS data [66], and Sichitiu evaluates an algorithm much like iterative multi-lateration using parameters derived from wireless nodes [90]. However, as we will see in the next section, the data sets from which the parameters are derived are not verified to conform to the Noisy Disk model Ranging Characterization Studies Almost all studies that characterize a new range sensor, shown in column 4, do so in terms of both Gaussian noise and Unit Disk connectivity (columns 6 and 8). Most such studies, however, do not use formal statistical tests to validate that the empirical data actually conforms to the Noisy Disk model (columns 5 and 7). Only two ranging characterization studies do not estimate standard deviation of noise. The first does not characterize noise at all, although it does state that the data appears to be Gaussian distributed [78]. The second is only characterizing connectivity [10]. Similarly, five studies do not explicitly assume the Unit Disk model of connectivity by estimating maximum range, but also do not propose a better model of connectivity. Instead, they do not characterize connectivity at all. Most of these studies do implicitly assume disk-like radio connectivity, however, by fitting the empirical data to a RF attenuation model of the form RSS = a b 10log 10 (r) (2.13) 1 These simulations used a variant of the Noisy Disk that assumes uniform, not Gaussian, noise. 2 All nodes in these deployments were within a single hop, and so key aspects of the algorithms may not have come into effect.

36 25 Patwari is the only study that attempts to verify the Gaussian noise assumption by plotting the data in a normality plot, which visually identifies deviations from the Normal distribution [67]. This shows that the radio signal strength data collected in this study appears to be Normally distributed. However, all of the data was collected with a single transmitter, a HP 8644A signal generator, and a single Berkeley Varitronics Fox high-fidelity WiFi receiver. The effect of other transmitters and receivers on the noise distribution is therefore unknown, and the results of this verification are therefore indicated in Table 2.1 as inconclusive. Bulusu is the only localization study that attempts to verify the Unit Disk model of connectivity by placing a 418MHz Radiometrix radio transmitter in the corner of a grid and measuring packet reception rates at the other grid positions [10]; 68 of the 78 grid positions measured exhibited packet reception rates that matched the predictions of the Unit Disk model. All 67 grid positions within range d max were measured and only one of these exhibited lower than expected packet reception. However, only 11 of the 33 grid positions beyond d max were observed, and 9 of these 11 produced higher than expected packet reception. The other 22 grid positions, if measured, may or may not have verified the Unit Disk model of connectivity. Furthermore, this experiment was performed with a single transmitter/receiver pair, reusing the same receiver node at every grid position. Therefore, this result is listed as inconclusive in Table 2.1. Later experiments in this study do appear to have enough data to either confirm or deny the Unit Disk model of connectivity, although an analysis of the data is not provided. Other studies from the wireless networking community that characterized similar radios indicate that the assumptions of Unit Disk model of connectivity do not hold [23, 101]. Unlike earlier studies that use a single transmitter/receiver pair, Ganesan uses 147 different nodes in a grid formation and each node acts as both a transmitter and receiver and Zhao uses a single transmitter and up to 60 different nodes in a line as simultaneous receivers. Both of these studies verify nearly complete connectivity at short distances and nearly no connectivity at large distances, but also demonstrate the existence of a large transitional region in between, in which levels of connectivity can be highly variable [104]. These studies show that the transitional region can occupy over 50% of the radio range, directly contradicting the assumption of Unit Disk model that the transitional region is negligible. The reason the transitional region is so large was not conclusively identified in these studies, although independent studies have documented significant differences between separate transmitters and receivers in radios [34] and ultrasound [95]. This factor is commonly believed to be at least part of the cause of non-disk like connectivity when multiple transmitters and receivers are being used in the same experiment.

37 Localization Deployment Studies Studies that actually perform a localization deployment can compare the observed localization error with that predicted by simulation for the purpose of top-down model validation. However, very few localization deployments have been performed and, of those, even fewer are compared with predictions from simulation. In most cases, the deployments are small enough that key aspects of the multi-hop localization algorithms are not playing a key role. These cases are noted in Table 2.1 with a footnote. For example, some deployments used four anchor nodes and only a single mobile node [10,45,83]. Other deployments used multiple nodes all within ranging distance of each other, forming a fully connected graph for localization [25, 66]. Moore [60] placed all nodes in a cell approximately the diameter of the maximum range, and the range of the ultrasound device was artificially restricted in software to limit the number of ranging neighbors that each node could obtain. This artificial restriction hid the effects of any natural deviation of the range sensor from the Unit Disk model of connectivity. Furthermore, in these experiments the range sensors were operating over only very short distances, which is the most consistent region of operation in terms of noise. Two of the three deployments that did use multi-hop topologies did not compare the results with predictions from simulation [28,48]. However, at least one of these still constitute strong evidence of the Prediction Gap. Kwon et al. estimated the maximum range of their acoustic sensors to be 22 meters, and accordingly placed 45 nodes in a grid with 9.14 meter spacing, a relatively close distance that would be expected to produce many range estimates and high node degree [48]. After ranging between all nodes, however, only 35% of the expected ranging estimates were obtained. This is far less than predicted by the Noisy Disk model and casts doubt on the Unit Disk model of connectivity. Because of this, the empirical deployment produced poor localization results and the authors needed to augment the observed range estimates with simulated estimates in order for the localization algorithm to work. To our knowledge, Stoleru et al. have produced the only study besides our own that evaluates a localization algorithm in both simulation using the Noisy Disk model and on a physical sensor network [92]. This study did not evaluate ranging-based localization algorithms; it evaluated both the Probability Grid and APS DV-Hop connectivity-based algorithms [63]. Even though the authors hand-calibrated the radios to make the empirical connectivity characteristics as ideal and disk-like as possible, a comparison revealed that the empirical localization error was up to eight times worse than predicted by simulation. The authors do not explain this discrepancy in the study,

38 27 and indicated that it requires further research. Nonetheless, this result is a strong indication that the Noisy Disk model does not withstand the tests of top-down validation, and provides concrete evidence for what we call the Prediction Gap.

39 28 Chapter 3 Establishing the Prediction Gap On the way to identifying why there is such a large Prediction gap in sensor field localization, we must first establish the gap ourselves. In this chapter, we establish the Prediction Gap by implementing and deploying a distributed, multi-hop localization system and comparing observed localization error with predictions by the Noisy Disk model. Our implementation builds upon and improves some of the best hardware designs and algorithms from existing systems to create a unified system that is designed specifically for the sensor field localization problem. The goal of this system is not to innovate in the area of localization, but rather to incarnate the canonical system that underlies the theory and assumptions in the multi-hop, ranging-based localization literature. We describe each stage of the system design and implementation, including hidden challenges and necessary innovations. In Section 3.1 we describe the ultrasonic ranging hardware, which combines ideas from several existing ultrasound implementations. We then describe a nonlinear noise filter in Section 3.2 that is designed to reduce the asymmetric noise profile of ultrasonic ranging. In Section 3.3, we describe our calibration techniques. In Section 3.4, we innovate a new collision detection scheme to eliminate error due to ultrasound collisions. We describe the localization algorithm in Section 3.5, and techniques for a distributed implementation of it in Section 3.6. In Section 3.7, we describe the incremental process of developing and debugging this system through simulation, small wired testbeds, and finally real world test environments. In Section 3.8, we deploy this system using a 49 node network. In Section 3.9, we compare the observed localization error with that predicted by the Noisy Disk model of ranging. To ensure a fair comparison, our deployment takes place in an ideal, open outdoor environment, we use the same topology for the deployment and simulation, and we derive the parameters for our

40 29 deployment and simulations from the same empirical data set. 3.1 The Ranging Hardware The first step to localizing our entire network is to design a simple range sensor that can estimate the distance between two nodes. Our ultrasonic ranging hardware combines and improves ideas from several previous ultrasound implementations. Our ultrasonic transducer circuitry is derived from that of the Medusa node [82], which uses 8 ultrasound transducers oriented at different angles, 4 for transmission and 4 for reception. Our circuitry is similar to the Medusa except that we add a switchable circuit so that a single transducer can be used to both transmit and receive. Our nodes measure ultrasonic time of flight by transmitting the acoustic pulse simultaneously with a radio message so that receivers can measure the time difference on arrival (TDOA) as described in Cricket [70]. When the transducers are face to face, our implementation can achieve up to 12m range with less than 5cm standard error. Comparable implementations were able to achieve proportionally similar results of 3 5m range with 1 2cm accuracy [61, 82, 83]. The differences in magnitude are due in part to our design decision to reduce the center frequency of the transducer from the standard 40kHz to just above audible range at 25kHz, which increases both maximum range and error. Ultrasound transducers are highly directional, and small variations from a direct face-toface orientation can have large effects on error and connectivity. Two solutions have been proposed to use ultrasound in multi-hop networks: aligning multiple transducers outward in a radial fashion [83] or using a metal cone to spread and collect the acoustic energy uniformly in the plane of the other sensor nodes [61]. We implemented the latter solution as shown in Figure 3.1 in order to avoid possible variations in range at different angles from the transducers. In this configuration, our nodes achieve about 5m range and 90% of the distance estimates are within 6.5cm of the true distances. The ultrasound transducer is connected to an Atmel Atmega8 1MHz micro-controller which is used for both transmitting and receiving ultrasound signals. The output of the transducer is wired to the analog comparator on the micro-controller for detecting incoming signals through simple threshold detection, and the value of the threshold can be controlled in software through a digital potentiometer. The input of the transducer is wired to a pulse width modulator (PWM) on the Atmega8, which directly keys the 25KHz signal. Both the transducer and the micro-controller are mounted as a daughter board which is attached to the Mica2Dot mote. Because the radio and ultrasonic transducer are controlled by different micro-controllers, a single interrupt line is used for precise time synchronization between them and the two micro-controllers exchange timing and

41 Figure 3.1: The Ultrasound Ranging Hardware is shown here. The white enclosure contains a mica2dot and battery and supports a reflective cone above the ultrasonic transducer, which protrudes from the top. 30

42 Time of Flight (msec) True Distance (cm) Figure 3.2: Raw Time of Flight Readings shown here were collected using our ultrasound hardware and the data collection process described in Section ranging information through an I2C communication bus. We characterized this hardware by collecting time of flight readings at multiple different distances using multiple different pairs of nodes in a data collection technique described in more detail in Section 4.2. The raw data is shown in Figure Noise Filtering Any technique for range estimation is susceptible to noise, which can often be reduced or eliminated by filtering a series of successive ranging estimates that are taken consecutively. For example, Figure 3.3 shows our raw ToF data after averaging each series of 10 consecutive readings. Averaging the data significantly reduces noise. However, the mean is known to be highly susceptible to outliers; data points extremely far from the mean can skew the mean until it is no longer representative of the series. Such outliers are common in ultrasound ranging. The type of noise found in time of flight range estimates is very structured, as illustrated by the time series of range estimates shown in Figure 3.4 that were taken between two nodes that

43 Time of Flight (msec) True Distance (cm) Figure 3.3: Averaging Ranging Data over a time series that is collected at the same time can significantly reduce noise with respect to the raw ToF data. However, non-gaussian outliers make the mean less effective than the non-linear filters demonstrated in Figure 3.5.

44 Raw Distance Estimates Filtered Distance Estimates Filtered Time of Flight Estimates Outliers 75 Distance Estimate (cm) Normal Noise False positives Time (sec) Figure 3.4: The MedianTube Filter chooses the minimum value within a small range of the median. This eliminates outliers and false positives and exploits the fact that the signal is often detected just after, and rarely just before, it arrives. are one foot apart. First, the data has outliers on both the positive and negative end of the noise distribution. The negative outliers are false positives; they represent detections of ultrasound before the ultrasound actually arrived, possibly due to ambient noise. The positive outliers are detections that do not occur until well after the signal has arrived, possibly because the incoming signal has a low amplitude. Besides the outliers, the rest of the points are fairly well concentrated. Due to the nature of time of flight, the lowest of these values is most likely to be the correct distance estimate; an ultrasound pulse is very likely to be detected shortly after it actually arrives, but is very unlikely to be detected very shortly before it arrives. Because of this asymmetry with ultrasound, we use a filter called mediantube, which reduces a time series of successive ranging estimates to be the smallest value within some pre-defined range of the median value of the series. In other words, the filter first removes outliers which are too far from the median value, and then chooses the smallest of the remaining values. The result of the mediantube filter on the data in Figure 3.4 with a sliding window of 20 samples is illustrated with a dashed line. Even though the median is known for being very robust to a small number of outliers, we

45 34 found through testing that mediantube performs significantly better than a simple median. This is because the median cannot detect when a series of readings is dominated by outliers due to a noisy signal between two nodes. On the other hand, when the mediantube filter identifies most readings from a series to be outliers, the entire series can be eliminated. Figure 3.5(a) shows the data from Figure 3.2 after the series of range estimates from each transmitter/receiver pair has been filtered using a simple median, and Figure 3.5(b) shows the same data after using the mediantube filter. 3.3 Calibration Calibration is the process of forcing a system to conform to a given input/output mapping. This is often done by adjusting the physical devices internally but can equivalently be done by passing the system s output through a calibration function that maps the actual device response to the desired response. For our localization system, the actual response is the ultrasonic TOF t ij between the transmitter i and receiver j and the desired response is the distance d ij. The calibration function must therefore be of the form d ij = f(t ij,β) (3.1) Where β R p are the parameters that describe the system. As described in Section 2.3.2, sound travels at a constant rate and multiplying ToF by the constant value of approximately 340 should convert the time of flight in seconds to the distance traveled in meters. Our calibration function must be slightly more complex, however, due to several other factors that affect TOF: 1. A non-zero delay δ between the transmission time of the radio message and the ultrasound pulse changes the measured ToF by a constant factor. 2. The times τ T and τ R required for the diaphragms of the transducers to begin oscillating during transmission and reception is non-zero, and add two constant factors to the measured ToF. 3. The volume of a transmitter V T and the sensitivity of the receiver S R affects the speed with which the signal can be detected. This latency can be incorporated as a multiplying coefficient of ToF because volume decreases with distance. 4. Received volume is also affected by signal attenuators in the environment such as grass or carpet and signal reflectors such as walls.

46 Time of Flight (msec) True Distance (cm) (a) Median Filter Time of Flight (msec) True Distance (cm) (b) MedianTube Filter Figure 3.5: The Effect of Filtering on our raw ToF data is shown here using a) a simple median filter and b) the mediantube filter. The mediantube filter is much more effective because it can identify a series of data that demonstrates little self-consistency.

47 36 5. A difference in transmission frequency and the receiver s center frequency, f T f R, has a near-linear affect on the effective received volume. 6. The relative orientations of the sounder and microphone, Φ T and Φ R, will affect the volume with which the acoustic tone is received according to some non-linear function f O ( ). We therefore arrive at the following complete model of the system response for a transmitter/receiver pair: ˆd ij = δ + τ T + τ R + V T t ij + S R t ij + f T f R t ij + f O (Φ T,Φ R ) t ij +Attenuation env t ij (3.2) To simplify this equation, we collapse all additive terms such as τ T and τ R into a single parameter β 1 and all multiplicative terms into a parameter β 2. Thus, all physical aspects of our system can be modeled by a linear calibration function d ij = β 1 + β 2 t ij (3.3) The exact coefficients can be estimated from empirical data to capture average node orientation, environmental influence, etc. To calibrate the ToF readings shown in Figure 3.5(b), each measurement is combined with the true distance at which it was measured using the equation above. We can combine all such equations to form a fully constrained linear system d ij = β 1 + β 2 t ij (3.4) d ik = β 1 + β 2 t ik (3.5) d jk = β 1 + β 2 t jk (3.6)... which can be trivially solved for β 1 and β 2 using least squares. The distance estimates produced from the ToF readings after calibration are shown in Figure 3.6(a). Because a single set of parameters are being used for all nodes, this process assumes that all nodes are the same. Any variations in transmitter strength, receiver sensitivity, or relative orientations are not captured individually by the parameters, which only represent the average value of all such variations. Therefore, we call this process uniform calibration.

48 Estimated Distance (cm) True Distance (cm) (a) Uniform Calibration Estimated Distance (cm) True Distance (cm) (b) Joint Calibration Figure 3.6: The Effect of Calibration on mediantube-filtered data is shown here using a) uniform calibration and b) joint calibration. Joint calibration does not do significantly better than uniform calibration, indicating that node variability is low.

49 38 In other work, we show that a different parameterization can be used with the same data set to estimate linear coefficients for each transmitter and receiver individually [96]; we must use a different set of parameters for each node i and j to create equations of the form d ij = βi t 1 + βj r 1 + βi t 2 t ij + βj r 2 t ij (3.7) instead of Equation 3.3. Because these coefficients represent the volume and sensitivity of each transducer individually, the resulting range errors have been shown to be significantly lower for some data sets. We call this process joint calibration. However, we found that joint calibration does not significantly affect our ultrasound ranging data set, indicating that there is only a small degree of variation between individual nodes with our ultrasound hardware. The ToF data from Figure 3.5 is shown after joint calibration in Figure 3.6(b). 3.4 Dealing with Collisions In the previous sections, we describe our techniques for estimating the distance between two nodes. In this section, we address a problem that occurs when there are more than two nodes: the ultrasound signals from two transmitters can collide. Pure ultrasonic tones cannot be differentiated, and so an interfering tone from one transmitter will change the estimated TOF from the other transmitter, causing large ranging errors. One technique to avoid ranging errors due to collisions is to have each node encode a unique signature in the ultrasound pulse through frequency or amplitude modulation. The receiver can be assured that the signal has been received without collision if the signature can be accurately decoded. This even allows the reception of multiple ultrasound signals simultaneously if the signature can be accurately decoded in the presence of interference from other transmissions [26]. Another solution is to simultaneously send both the ultrasound pulse and a radio message, such that a radio message is always being sent if an ultrasound pulse is being sent. If the radio signal can be accurately decoded, the receiver can be assured that no other radio message, and therefore no other ultrasound pulse, was being sent simultaneously [70]. One advantage of this technique is that it reuses the modulator/demodulator on the radio instead of building redundant functionality on the acoustic transducer. It relies on the fact that radio collisions always result in corruption of all messages involved. Our ultrasound implementation initially employed the latter of these two techniques, the RF envelope, because of its simplicity; the ultrasound pulse does not need to be modulated or

50 39 Figure 3.7: Capture Experiment Setup consisted of three nodes in an equilateral triangle. Nodes A and B both transmit packets that overlap in time while the third node attempts to receive them. decoded, reducing both memory requirements and power consumption. However, collisions were still a problem because radio collisions do not always result in corruption; when two nodes transmit simultaneously with the FSK radio used on the mica2dot, a third node will sometimes receive one message completely uncorrupted, even if both are received with almost the same signal strength. This phenomenon is known as capture and can be the cause of very large ranging errors: if two nodes A and B transmit both RF and ultrasound signals that overlap in time and the RF signal from A is received clearly, but the ultrasound pulse from B arrives first, the TOF estimate will be an arbitrary value related to the distance of neither A nor B from the receiver. We performed an experiment of controlled collisions to measure how often this phenomenon might cause ranging errors by generating two radio packets with a precise time difference t. As illustrated in Figure 3.7, two transmitters A and B and a receiver are placed in an isosceles triangle and the two transmitters are synchronized to transmit at times t A and t B such that the time between them was t = t B t A. Thus, when t is positive A transmits first and when it is negative B transmits first. t is varied from 23ms to 23ms at 1ms intervals and 10 collisions were generated at each value of t. The packets are 17.9ms long, so 0.5ms intervals are used around

51 40 = 0ms and = ±18ms for higher resolution data. Time synchronization was only accurate to 1ms. While a 0dBm transmit power was used on both nodes, slightly moving one of the senders or adjusting the antenna orientation changed the power relationship between the two senders at the receiver. It was fairly difficult to find a null point at which neither transmitter was received due to the difference in received energy from the transmitters being below the SNR threshold of the receiver. In fact, we confirmed that the power difference required to cause one transmitter to be received over the other one was not observable using the 10bit ADC to sample the RSSI pin on the radio. In our experiments, we deliberately move node B such that its signal was stronger than node A at the receiver. Figure 3.8 summarizes the findings from this experiment. The Y axis is the percentage of packets received while the X axis is the t between packet start times. In the left half of the graphs B sends first while on the right half of the graphs A sends first. At the two edges of the graphs where t > 17.9ms the messages do not overlap in time and both are received without corruption. At 0 < t < 17ms, A is sent first and, because B is stronger, it corrupts the tail end of A s message when it arrives. However, when 17ms < t < 0, B is sent first and, due to the capture effect, is not corrupted by A at all once A is sent. Therefore, the message from B is received without corruption even though it overlaps in time with the message from A. This experiment shows that, with the CC1000 radios, the capture effect is quite common and can cause ranging errors up to 50% of the time. To remedy this situation, we implemented an application-level collision detection protocol in which each node sends ranging messages in batches of ten with a small random delay between each message. The maximum random delay is about 10 times the length of the packets, making the probability p m of two individual messages colliding about Therefore, the probability p b that every message in a batch from one node collides with every message in a batch from another node decreases exponentially as the length n of the batch grows, as p b = p n m Using batches allows receiving nodes to detect and discard data from ranging collisions with high probability: any node that hears messages from two overlapping batches can discard all ranging messages from both batches. Using these techniques, all nodes in a network can obtain range estimates to their neighbors in a single ranging phase, in which each node randomly sends ranging messages to all neighbors, which collect the range estimates, filter them, apply a calibration function, and observe the occurrence of collisions. In the next two sections, we describe how to use these range estimates to

52 Figure 3.8: The Prevalence of Capture is indicated in this experiment. Nodes A and B both transmit with a time difference of t. When 17ms < t < 0, the messages overlap but a third node can still hear B s message with no corruption. 41

53 42 derive the locations of the nodes. 3.5 The Localization Algorithm After the ranging phase is complete, the network contains the ranging graph G described in Section 2.1, with distance estimates between each node and its neighbors. This graph must be used in the localization phase to derive the positions of the non-anchor nodes. To localize the nodes in our system, we implemented a decentralized version of the Adhoc Positioning System s (APS) DV-distance algorithm [63] using TinyOS [55] and nesc [24]. DVdistance is only one of many ranging-based algorithms that have been proposed for localization, but we use it in this study because it represents a large class of algorithms that use shortest-path distance to estimate true distance [80, 82, 87, 91, 92]. DV-distance has also been shown to yield comparable localization error to the other distributed localization algorithms [50]. DV-distance estimates the distance between a node and an anchor to be the sum of the distances on the shortest path through the network between them. These shortest path distances are then used by each node to solve the linear system in Equations 2.4, effectively reducing the multi-hop localization problem to single-hop localization. To find the shortest path distances, DV-distance uses a distributed distance vector algorithm. Before the algorithm begins, each node must have range estimates to all of its neighbors and must initialize its shortest path distance to each anchor to be. Each anchor node initiates a run of the algorithm by broadcasting a shortest path (SP) message with the following information: the source node ID i the anchor node ID j the anchor node location the shortest path distance estimate sp ij from i to j When the anchor node initiates the algorithm, i and j are set to the anchor s ID and the shortest path estimate is 0. Whenever a node k hears a SP message, it compares its own shortest path estimate sp kj to the sum of the shortest path from the message sender and the range estimate to the sender sp ij + d ki. If the latter is shorter than the former, the node updates its shortest path estimate and sends a new SP message with the updated information. In this way, a single message from an anchor

54 43 triggers an iterative algorithm through which all nodes acquire the anchor node s position as well as a shortest path estimate to that anchor. Shortest path distances, of course, are only estimates of the true distance between a node and an anchor node; they must route around non-convex network topologies and their zig-zag nature should always make them longer than the true distance. APS uses a correction factor to correct for such systematic biases, exploiting the fact that anchors know both the true distance to other anchors and the shortest path distance estimate, by sending the ratio of these to all nodes surrounding each anchor. A node i near anchor j that receives a correction factor for anchor k can multiple its own shortest path estimate to k by the correction factor. This is intended to remove systematic bias from shortest path estimates, assuming that the same factors that affect the shortest path from j to k also affect the shortest path from i to k, although there is no guarantee. 3.6 Distributed Programming Sensor field localization can be seen as a distributed programming problem; each node must perform local operations using data that is resident on other nodes in the network. However, distributed programming on a sensor network can be difficult, and we require new programming abstractions to create a DV-distance implementation that can run on sensor nodes. Traditional abstractions like distributed shared memory or tuple spaces are difficult to apply to sensor networks because of the unreliable, bandwidth-limited, geographically constrained communication model. Furthermore, these traditional abstractions are not necessary for most sensor algorithms, which are typically based on local communication among neighboring nodes; each node selects a subset of the nodes within radio range, maintains state about them, and shares data with them. However, this concept of a node and its neighborhood is still not a programming primitive in the sensor network community. Neighborhood-based algorithms are typically implemented as compositions of other more primitive parts such as neighbor discovery, data caching, and messaging protocols. This can make programming a distributed application like localization challenging. To facilitate this process, we define a concrete relationship between these concepts in a single unified programming abstraction called Hood, which allows developers to think about and implement algorithms directly in terms of neighborhoods and data sharing instead of decomposing them into lower-level programming abstractions. A neighborhood is defined with Hood by a set of criteria for choosing neighbors and a set of variables to be shared. For example, Hood can define a one-hop neighborhood over which

55 44 light readings are shared and a two-hop neighborhood over which both locations and temperatures are shared. Once the neighborhoods are defined, Hood provides an interface to read the names and shared values of each neighbor. Beneath this interface, Hood is managing discovery and data sharing, hiding the complexity of the membership lists, data caches, and messaging. Attributes are the elements of a node s state that are shared with its neighbors, such as sensor readings or geographic location. When a node updates its own attribute, the value is said to be reflected to its co-neighbors, much like traditional reflective memory (RM) [89]. Exactly how data is reflected is determined by the push policy. Typically, this is simply to broadcast the value once each time it is set, but could also be to broadcast periodically or reliably. When an attribute is received at a co-neighbor, it is passed through the filters of each neighborhood defined on that node. Filters examine each shared attribute to determine which nodes are valuable enough to place in the neighbor list and which attributes of those nodes need to be cached. For each node in the neighbor list, a mirror is allocated, which represents the local view of that neighbor s state. It contains both reflections, which are cached versions of that neighbor s attributes, and scribbles, which are local annotations about that neighbor. Scribbles are often used to represent locally derived values of a neighbor such as a distance estimate or link-quality estimate. A node can define multiple neighborhoods with different variables shared over each of them, although the members of each neighborhood may overlap. Figure 3.9 shows an example of a node that is sharing its Mag and Location attributes. It defines two neighborhoods: the Tracking Neighborhood consists of three nodes that have Mag values that exceed some threshold, and with which it shares both the Mag and Location attributes. The Routing Neighborhood consists of the eight nodes that are closest geographically, and with which it shares the Location attribute. The Receive Link Quality and Transmit Link Quality are scribbles that are maintained locally about each node in the Routing Neighborhood. All data sharing and data caching in our implementation of the DV-distance algorithm is taken care of by the Hood abstraction. We define a Ranging Neighborhood containing all neighbors to which a node can obtain a valid range estimate. The mirror for each neighbor contains its series of range estimates, calibration coefficients, and the result of the mediantube filter for that neighbor. The Anchor Neighborhood contains the four nearest anchor nodes, and the mirror for each anchor contains its location, a shortest path estimate to that anchor, and up to three anchor corrections from that anchor node. When a shortest path update message arrives from a neighboring node, the neighborhood manager checks the Ranging and Anchor neighborhoods to see if a local shortest path estimate needs to be updated. If so, the new shortest path information is added to the neighborhood

56 45 Tracking Neighborhood Attributes Routing Neighborhood Mag Nbr 1 Mag Nbr 2 Mag Nbr 3 Rte Nbr 1 Rte Nbr 2... Rte Nbr 8 Mirrors Mirrors - - Re - r - Mag Mag Mag Mag - - R R Rx Link Quality... r - Location Location Location Location - R Location Refl Location - R Location S Rx Link... Rx Link Quality Quality Quality Quality... - S Tx Quality Mirrors Mirrors Mirrors Mirrors Figure 3.9: The Hood Programming Abstraction provides a high-level interface for neighborhood-based data sharing. In this figure, the Tracking Neighborhood on the left contains three nodes with which this node shares Mag values and Location values. The Routing Neighborhood on the right contains eight nodes.

57 46 and automatically shared with all other neighbors. 3.7 Implementation and Debugging Our implementation of the DV-distance algorithm runs in four fully decentralized phases. To initiate each experiment, the network is flooded with parameters such as transmission power and calibration coefficients. The four nodes in the corners of the network are designated as anchor nodes and are given their true positions, at which point they initiate a ranging phase in which all nodes estimate the distance to each of their direct ranging neighbors. The anchors then initiate a shortest path phase, in which anchors initiate the distributed shortest path algorithm described above. Then, the anchor nodes initiate a anchor correction phase, in which anchor correction factors are broadcast in a regional flood. When all flooding is complete, each node estimates its own position in the localization phase. The phase transitions are initiated by the anchor nodes, which listen to network traffic to determine when each preceding phase is ending. From the time that the anchor nodes are given their positions, the entire process is automated with no human intervention or central computer and completes in less than five minutes for each deployment. During each experiment, a laptop eavesdrops on the network to reveal current progress and, afterward, an automated script retrieves all ranging estimates, shortest paths, and estimated locations that were stored in RAM on the nodes. Development of this system took place in several phases. First, it was debugged on a large scale using the TOSSIM simulator for TinyOS and the TinyViz visualization component [54]. The visualization in Figure 3.10(a) shows the system on a 10-node network, where the blue arrows indicate localization error vectors, which is the difference between the estimated position and the true position. The grey circles indicate estimated error. These simulations were scaled up to 150 nodes. Subsequently, the algorithm was programmed onto Berkeley s Mica2Dot mote [35], which consists of a ChipCon CC1000 FSK 433Mhz radio and an Atmel Atmega128 4MHz micro-controller and was equipped with the ultrasound hardware described in Section 3.1. Because development was not taking place in a simulated environment, we made most of the functions and variables in our code remotely accessible from the PC through the Active Message interface. We could access these functions and variables as well as reprogram the nodes through a wired testbed that we built using the Crossbow EPRB programming device and an Ethernet switch, as shown in Figure 3.10(b). However, because the testbed was not mobile and was in a space-confined location, these tests were limited to localization with a maximum of 12 nodes. Once the system worked on this wired, in-

58 47 (a) Simulation The localization implementation was first tested and debugged in a simulated environment, scaling to 150 nodes, using TOSSIM simulator and the TinyViz visualization tool, shown here. (b) Indoor Testbed The localization system was first tested in real hardware on the wired testbed shown here, which allowed debugging commands and reprogramming of all nodes on the network while simultaneously testing the code with real hardware. Figure 3.10: Stages of Development and Debugging include simulation and small, wired testbeds.

59 48 Figure 3.11: The Final Deployment involved 49 nodes over a 13x13m area on a paved surface. door testbed, we tested the system in an outdoor environment shown in Figure 3.11, first in a 16- node topology and scaling upwards to 25-, 36-, and 49-node topologies. In this environment, we used the wireless communication channel for both application data and debugging commands, and reprogramming of the network was performed manually. 3.8 Deployment Details In our final deployment, 49 nodes were deployed over a 13x13m area in a 7x7 grid, in which each of the grid positions was perturbed by Gaussian noise with σ = 0.5m. We used a randomly perturbed grid to avoid artifacts of the rigidity of a strict grid or the network partitions common in completely random topologies. To avoid performing this one deployment with a topology on which our system would by chance perform unusually well or badly, we generated 100 random topologies, simulated the algorithm on each of them, and chose to use the random topology which yielded the median average error in simulation. The longest shortest path in the selected topology was eight hops long, and even longer paths appeared in the real deployment. The main deployment took place outdoors in the parking lot shown in Figure We

60 49 Probability Empirical Error Distribution (kernel smoothing) Error (cm) Distances in Centimeters Figure 3.12: Localization Error Vectors are shown in this graph by arrows; the true node positions are the beginning of each arrow. The anchor nodes are indicated by X s and the gray lines indicate ranging connectivity. Nodes 33, 16, and 43 were dead nodes. The median error for this run was 47.8cm, and the top graph is a kernel smoothing of the error distribution. used a system of tape measures to deploy the nodes with an estimated accuracy of about 2.5cm. After we measured the topology and placed the nodes, we executed the localization system on the network eight times. The median localization error for this deployment was 78.0cm, while the upper and lower quartiles of error were 131.2cm and 40.5cm, respectively. The actual topology used in this deployment can be seen in Figure 3.12, along with the localization errors resulting from one of the several runs of the localization algorithm.

61 Upper Quartile, Median, and Lower Quartile 200 Localization Error (cm) Noisy Disk Deployment Figure 3.13: The Localization Error Gap is illustrated by the dramatic difference in localization error predicted by the Noisy Disk model in simulation and that observed in the real deployment. The box indicates the median localization error and the error bars indicate the 10th and 90th percentiles.

62 Comparing Theoretical and Observed Localization Error We reused the topology from our empirical deployment in simulation to see how the observed localization error compares to the localization error predicted by the Noisy Disk model of ranging. To derive the Noisy Disk parameters for simulation, we used the ranging data that was collected in the same location as the deployment, which is also the data set that was used to derive the calibration coefficients for the empirical deployment, as described in Section 3.3. We used the value d max = 450cm based on the maximum distance that our ranging technology could robustly reach, as demonstrated by the data in Figure 3.6(a). We used the value σ = 4.9cm based on a maximum likelihood fit of the nominal ranging errors. We excluded the outliers from this fit by fitting the ranging error to a Gaussian mixture model with 2 means, and choosing the value σ with the highest posterior probability. If the dataset contained no outliers, this procedure would yield the same value σ as the standard maximum likelihood estimator. Even though the Noisy Disk parameters were derived from the data in a way similar to derivations from previous work described in Section 2.5.2, the localization errors in simulation were significantly different than those observed in the empirical deployment. The median localization error in simulation was 15.2cm, with upper and lower deciles of error of 34.7cm and 6.0cm, respectively. This is significantly lower than the error distribution actually observed in the empirical deployments, with a median of 78.0 and upper and lower deciles of 208.3cm and 23.2cm. Both error distributions are shown in Figure A deeper analysis reveals that the simulation differs from the deployments not only in terms of localization error, but in terms of intermediate values that are generated, as well. For example, we can characterize shortest path error as a ratio of the shortest path distance to the true distance between two nodes. If the shortest paths are all exactly correct, this ratio should be exactly 1. The shortest path error for both simulation and deployment are similar, but the variance of the shortest path error is much higher in the deployment than predicted by simulation. While the center 80% of the shortest path errors vary by less than 3% of the true distance in simulation, they vary by up to 19% in the empirical deployment. The shortest path error distributions are illustrated in Figure In localization, a node s degree is the number of neighboring nodes with which it can obtain a range estimate. Figure 3.15 shows that the median node degree in simulated localization runs is 12 while in the empirical deployment it is only 6. This difference can be extremely significant; studies have shown that localization algorithms can behave very differently in high density networks

63 Upper Quartile, Median, and Lower Quartile 1.15 Shortest Path Error (cm) Noisy Disk Deployment Figure 3.14: The Shortest Path Error Gap is primarily exhibited by a difference in variance. In simulation, shortest paths vary by 3% while in the real deployment they vary by 19%.

64 53 18 Upper Quartile, Median, and Lower Quartile Node Degree Noisy Disk Deployment Figure 3.15: The Node Degree Gap indicates that nodes in simulation have on average 12 ranging neighbors while in the real deployment they had only 6.

65 54 than in low density, where the threshold between the two is approximately a node degree of about 9 [50] The Prediction Gap Established In this section, we provided concrete evidence of the Prediction Gap. The localization system we designed was representative of the canonical system used in most localization simulations and theoretical analysis in the literature. It was carefully designed to provide range estimates that are as good or better than most existing implementations, and we provided analysis and solutions for problems that were previously not addressed, including collision avoidance and non-linear noise filtering for the asymmetric noise profile of time of flight ranging. The localization algorithm we used is representative of a large class of existing localization algorithms, and has been experimentally shown to produce comparable localization results and to have similar failure modes. We extended the algorithm by carefully building a distributed implementation, which required new programming abstractions and an incremental development process through simulation, emulation, small wired testbeds, and ultimately real deployments. By combining range sensors and a distributed localization algorithm, this system is a precise and complete representation of many canonical ideas from the localization literature, and the unexpected performance observed in Section 3.9 cannot be attributed to implementation or design issues, but rather to a lack of understanding in the literature of how such a system behaves. Similarly, our comparison with the Noisy Disk model was preceded by a very careful analysis and thorough characterization of our range sensor. The characterization captured aspects of many different transmitter/receiver pairs at random orientations and in multiple different paths through the actual deployment environment. We captured an abundance of data at a high resolution: at least one point every 2.5cm over the entire range of the sensor. This data set was used to set the calibration and filtering coefficients in our deployment, and the same data set was used to estimate the Gaussian noise and Unit Disk parameters that we used in simulation. Thus, differences between our simulation results and observed deployment results cannot be attributed to using an unrealistic simulation scenario. Rather it should be attributed to a failure of the Noisy Disk model to capture the structure of our empirical ranging data. This argument is supported by the fact that our comparison corroborates previous studies which found that simulation does not accurately predict true deployments. In the next chapter, we will take this analysis one step further by identifying the cause of the Prediction Gap that we observed in this chapter.

66 55 Chapter 4 Closing the Prediction Gap The previous section shows that traditional simulation of localization using the Noisy Disk of ranging model does not accurately predict the localization errors observed in the empirical deployment. This difference is what we can been calling the Prediction Gap, and is a long-standing problem in the localization literature for three reasons: 1. Real deployments are unpredictable. If an application such as tracking specifies a maximum allowable localization error, a real deployment may not meet that requirement even if it is predicted to do so in simulation. This can be a problem for mission critical deployments which can only be deployed once, such as forest fire tracking, or for large deployments with 1000 s of nodes where the cost of redeployment is prohibitive. 2. Comparison of algorithms is inconclusive. Besides predicting the localization error of a particular real deployment, simulation is also used to compare algorithms and to analyze the sensitivity of an algorithm to different noise levels or topologies. Because there is no concrete relationship between simulation and empirical deployment, the conclusions from simulationbased analysis may not hold in the real world. 3. Empirical error is difficult to explain. If everything known about the environment and range sensor is incorporated into a theoretical model which produces low errors in simulation, then the cause of any additional error observed in the real deployment is not known. Furthermore, if the cause of the additional error is not know, it is difficult to reduce. The first step to addressing these issues is to reevaluate our model of the sensors and the environment and to create one that accurately predicts empirical localization error. There are

67 56 several challenges to improving the traditional parametric Noisy Disk model. Instead, we choose to use non-parametric models, which take data collected in the real world and use it directly in simulation, avoiding the need to reduce complex empirical data to a simple set of parameters. This can produce accurate simulations without committing in advance to a particular parametric form of the empirical data. 4.1 Modeling the sensors and environment Parametric Models Parametric models like the Noisy Disk specify a structure that can only change in a certain number of ways, as enumerated by the model s parameters. Many techniques have been developed for choosing the best parameters to fit a model to a data set. For example, least squares fitting chooses the parameters that minimize the squared difference between the observed data points and predictions from the model [65]. Robust estimation techniques are similar, but they place lower weight on points that are not well predicted by the model [37]. Maximum likelihood techniques maximize the probability of the data points given the parameters [73]. Indeed, most machine learning techniques, including neural networks [77], the expectation-maximization (EM) algorithm [73], and support vector machines [16] are all parameter estimation techniques that assume the user has already determined the general structure of the data in some parametric form. In contrast, very few formal techniques exists in the way of choosing the model itself. This is a natural dichotomy because a model defines a clear parameter space, but the space of all models is typically not well defined. Defining the space of all models would require a neighborhood function that defines a transformation from one model to other similar models and creates a well-behaved space of models over which an algorithm may search. This is possible in some cases such as neural networks and Bayesian networks with structures that facilitate search using genetic algorithms [3] or other techniques [33], even though the search over such structures has been shown to be NP- Complete at least in some cases [14]. Defining a good neighborhood function over algebraic models like Equation 2.6 to create a searchable space of algebraic formulae is much more difficult. Instead, when creating an algebraic model of a process such as ranging, scientists typically resort to first principles of physics; each aspect of the hardware, signal propagation, and environment are modeled according to algebraic formulae from traditional physics. In our case, the ranging process is captured by our model in Equation 3.2.

68 57 The problem with the technique of first principles is that it leaves us with no further recourse when we find our model to be insufficient. In general, the model is known to be a simplification of the real physical process; the calibration function we used in Equation 3.3, for example, does not explicitly account for the orientation or frequency variations of our original model in Equation 3.2 and instead incorporates these into the noise parameter, along with many other aspects of the physical world that are too complex to model. Improving our model requires us to identify which of these physical processes produce salient effects that cannot be treated as noise, or that change our assumed noise distribution. This task is made more difficult by the fact that a physical property of the range sensor and environment may affect one localization algorithm but not another and that this effect may be exhibited in one network topology but not another. Therefore, to create an improved model from first principles, the scientist must not only understand the physical world but also its complex interaction with network topology and the implicit assumptions of a particular algorithm. In our case, it is not immediately clear which aspects of our environment and range sensor are causing unexpectedly higher error in deployment than with the Noisy Disk Non-parametric Models Non-parametric models differ from parametric models in that the structure of the data is not assumed in advance, but is instead determined by the data being modeled. Non-parametric models are also called distribution free models because they do not assume the data conforms to some predetermined distribution. Several forms of non-parametric models exist, the most common of which include histograms [85], kernel regression [39], and wavelet analysis [43]. In this section, we show how to use statistical sampling, in which we generate data for simulation by randomly drawing measurements from an empirical data set. We define the distribution M(d,ǫ) to be the set of all observed ranging estimates for distances in the interval [d ǫ,d + ǫ]. This set is our non-parametric model and represents an empirical distribution of range estimates at distance d. For example, the set M(350cm, 5cm) is represented by all the range estimates between the vertical bars in Figure 4.1. We can generate a ranging estimate ˆd ij for simulation from this model by simply drawing a random sample d from the set M(d ij,ǫ). Using the value of d directly, however, would not be accurate; the value of ǫ increases the variance of ˆd ij because M(d ij,ǫ) includes range estimates from both longer and shorter distances than d ij. Instead, we use the error of the sample, which is the difference d d a where d a is the actual distance at which d was measured. Thus, a simulated

69 58 Figure 4.1: The Non-parametric Model is essentially a binning of empirical data. The blue dots indicate observed data points. All dots between the two red lines are binned into a set called M(350cm,5cm), which is randomly sampled to simulated d ij = 350cm in simulation.

70 59 error measurement can be generated from M(d ij,ǫ) as ˆd ij = d ij + ( d d a ) (4.1) Besides range estimates, the set M(d, ǫ) also includes ranging failures, denoted by ø, which are ranging instances when a pair of nodes fail to obtain a distance estimate. This is necessary to model the probability of connectivity at distance d; if 50% of all ranging estimates taken at distance d are ranging failures then randomly sampling from M(d,ǫ) should yield a 50% chance of drawing ø. 4.2 Empirically Profiling the Physical World Traditional Data Collection Because d M(d ij,ǫ), the simulation is using the empirical distribution of ranging estimates at distance d ij if and only if M(d ij,ǫ) accurately represents the noise and connectivity characteristics at that distance. The challenge in using this sampling technique is to collect ranging error and connectivity data with a high enough resolution so that small values of ǫ can be used. For example, if we want to use ǫ = 2.5cm and ultrasound ranging has a maximum range of 10m, we must take empirical ultrasound measurements at 400 different distances. The typical data collection process, however, makes it difficult to collect data with such high spatial resolution: one usually places a transmitter and receiver a known distance apart, collects range estimates, and repeats at a small number of increasing distances. The data set collected by Sichitiu [90] in Figure 4.2, for example, collects ranging data with up to 10m spacing. The low spatial resolution of this data would make it difficult to use values of ǫ smaller than 5 meters. Furthermore, to use this data set with our non-parametric model to simulate ranging at a distance of d = 25m, we would need to assume that data collected at d = 20m and d = 30m has roughly the same characteristics as d = 25m. The problem with the traditional data collection process is that it requires a linearly increasing amount of time as the number of distances are measured. Even if all readings can be taken in 60 seconds at each distance, measuring 400 different distances would require almost 7 hours. Not only does this make it difficult to collect samples from multiple different combinations of range sensors and environments, but it makes it impossible to collect a complete sample of an outdoor environment, for example, before the temperature, humidity, and wind conditions change. Of the authors mentioned in Section that collected empirical ranging data, most collected data at no more than 15 different distances.

71 Figure 4.2: Traditional Data Collection results are illustrated here, in which RSS data was collected at multiple different distances by taking a single pair of nodes and placing them at progressively larger distances [90]. Because of the low resolution, this data would be difficult to use with our non-parametric model. 60

72 61 Another problem with the traditional data collection approach is that it only measures a small number of points in the total space of noise factors: ranging data is measured with a single transmitter and a single receiver, usually in the same orientation, and in a single line through space. This means that any idiosynchracies of the particular transmitter and receiver are present in all data collected at all data points and any physical aspects of the testing environment, such as a wall several meters away or the orientation of the nodes, may produce systematic errors in the entire data set. Of the empirical data collection studies mentioned in Section 2.5.2, all authors collected data with a single transmitter and receiver High-fidelity Data Collection Instead of measuring each distance with a single pair of nodes, we designed a data collection process that could measure several hundred distances as well as different transmitter/receiver pairs, node orientations, and paths through the environment. All measurements are taken at once with 400 = 20 nodes in a special topology where each pair of nodes measures a different distance. By adding a few additional nodes, we can get multiple pairs of nodes at each distance. We generated such topologies using rejection sampling [74], i.e., we generated thousands of topologies until one of them measured a uniform distribution of distances. For example, we used the topology in Figure 4.3(a), which required 25 nodes to obtain 2.5cm resolution over 5m, to characterize our ultrasound range sensor. Figure 4.3(a) shows a histogram of the distances that are measured by this topology. All nodes are placed at random orientations in this topology and each node transmits N times in turn while all other nodes receive. To remove the bias of each distance being measured by only two pairs of nodes (the reciprocal pairs A/B and B/A), this procedure is repeated five times with different mappings of nodes to the topology locations. These mappings are also generated using rejection sampling to ensure that the same distances are not always measured by the same pairs. The procedure generates 10 N total measurements at each distance with 10 different transmitter/receiver pairs. Therefore, with the topology in Figure 4.3(a) and values N = 10 and ǫ = 0.05m (two inches), the set M(δ,ǫ) is likely to include 400 empirical measurements. Unlike the conventional pairwise data collection technique described above, the empirical measurements in M(δ, ǫ) are taken with dozens of transmitter/receiver pairs, capturing a broad spectrum of node, antenna, and orientation variability. Furthermore, the measurements are taken over several different paths through the environment, capturing variability due to dips, bumps, rocks

73 G B C L N W F P X E J Q 3 Y V D H 2 T U R A 1 I K S O M Distances are in meters (a) Topology This specially generated topology with 25 nodes measures 300 different distances with at least 1 distance every.025m between 0.4m and 5.2m. 8 Distribution of Measured Distances #Measurements Distance (m) (b) Histogram This histogram shows that the distances measured by the topology are uniformly distributed over the ultrasonic range. Figure 4.3: The Data Collection Topology

74 63 Indoors Grass Elevated Evening Max Range d max (cm) Noise σ (cm) Table 4.1: Generalizing Noisy Disk Parameters can be difficult, because the Noisy Disk parameters are very different for each of the different environments in Figure 4.4 or other environmental factors. Finally, this technique captures connectivity characteristics by fixing the number of transmissions and measuring the number of readings at each distance. In contrast, the conventional pairwise technique described above requires the experimenter to take readings at every possible distance, hiding the degradation of ranging connectivity with distance. The rejection sampling algorithms required on average twelve hours to compute the topology and node mappings. We measured the topology positions using tape measures by first measuring out two right triangles to create a square and then placing two tape measures along the vertical edges of the square and one which ran horizontally between them. To locate each position, the horizontal tape measure was slid up or down to find the correct Y coordinate, and the X coordinate was found on the horizontal tape measure itself. This process could be completed in about 1 hour and required two people, as opposed to the traditional process which requires only a single experimenter. For each mapping of nodes to topology locations, ranging between all nodes took place over a period of about 5 10 minutes. Another 20 minutes was required to collect the data to a central base station over the wireless network. The nodes were then collected and redistributed in a new mapping of nodes to topology positions. The data collection process extended over a period of about 3.5 hours, but could be reduced to about 1 hour if we stored the data to external flash on the nodes and retrieved it after the experiment, or if we used the faster radios that are now common in sensor networks for faster data collection Generality of an Empirical Profile Once data is collected in a particular environment, it can only be used to simulate a deployment in that same environment. We used the data collection process described above to collect data in several different environments, including indoors, outdoors in a grassy field, on pavement during the day and at night, and in a network raised above grass to approximate free space. Some of these environments are shown in Figure 4.4. Each of these environments yields data with very different characteristics, as becomes

75 64 (a) Indoors (b) Grass (c) Elevated (d) Evening Figure 4.4: Profiling Multiple Environments using the data collection techniques described in Section reveals that data collected in one environment may be very different than other environments.

76 65 evident when we fit the data with the Noisy Disk model using the approach described in Section 3.9. Note, for example, that the grass environment has a maximum range only 63% the length of the range in a parking lot, while the elevated nodes yield 37% more noise than the parking lot. Table 4.1 shows the maximum range d max and noise parameter σ derived from the data collected in each of the environments in Figure 4.4, showing how significantly different environments can affect the time of flight measurements. Although an empirical profile from one environment cannot be generalized to other environments, this is not a limitation only of non-parametric models. As described in Section 4.1.1, the parameters of parametric models must also be derived from empirical data and, as with nonparametric models, these parameters cannot be generalized to environments other than the one in which that data was collected. 4.3 Comparing Non-parametric Predictions and Observed Localization Error The empirical profile of our deployment environment was earlier used to derive calibration coefficients in Section 3.3 and to derive Noisy Disk parameters for simulation in Section 3.9. In this section, we use the same empirical profile as a non-parametric model of our deployment environment. Similar to our comparison between Noisy Disk simulation and the deployment, we reuse the topology from our empirical deployment in simulation to see how the observed localization error compares to the localization error predicted by the non-parametric model of our range sensor and environment. Because the empirical profile was collected with a resolution of approximately 2.5cm, we choose the parameter ǫ = 5cm for our model. The non-parametric model of our environment predicts the localization error from our true deployment much more accurately than the Noisy Disk model. This is demonstrated by Figure 4.5, which shows the error distributions for the deployments and both simulations. The Noisy Disk simulation predicts a median error of about 15, with upper and lower deciles of error of 34cm and 6cm, respectively. This is much smaller than the observed median error of 78cm, 90th percentile of 208cm, and 10th percentile of 23cm. Indeed, the predicted median error is lower than the observed 10th percentile. The non-parametric simulation produces a much more accurate prediction, with median error of 67cm, 90th percentile of 174cm, and 10th percentile of 22cm. The localization error distribution from the deployment is still significantly different than

77 Upper Quartile, Median, and Lower Quartile 200 Localization Error (cm) Noisy Disk Non parametric Deployment Figure 4.5: Closing the Localization Error Gap can be performed with non-parametric modeling, although all three distributions are still statistically different. Non-parametric simulation does not explain the shortcomings of the Noisy Disk model, but this will be performed in Chapter 5.

78 Upper Quartile, Median, and Lower Quartile 1.2 Shortest Path Error (cm) Noisy Disk Non parametric Deployment Figure 4.6: Closing the Shortest Path Gap with non-parametric simulation yields similar median shortest path errors and similar variance, although all three distributions are statistically different. the distribution from the non-parametric simulation, as determined by a two-sided t-test with α = However, almost no simulation technique can be expected to produce exactly the same error distribution as the real world, and the predicted results are qualitatively very close or at least, in contrast to the Noisy Disk simulation, represent the correct order of magnitude. The non-parametric model is not only a better predictor of the overall algorithmic behavior, but is also a better predictor of the internal structure of the algorithm. As shown by Figures 4.6 and 4.7, the non-parametric model more accurately predicts the distribution of shortest path distance errors and node degrees, respectively, than does the Noisy Disk model. The distribution of node degrees produced by the non-parametric simulation and the deployment are statistically equivalent, according to a two-sided t-test with α = 0.05 and p = The reason why the overall localization error and the internal algorithmic behavior is more accurately predicted by the non-parametric model is not immediately clear from these results; they only show that the non-parametric model is an improvement over the parametric model. This improvement will be more completely explained in Chapter 5, when we combine the two techniques

79 68 18 Upper Quartile, Median, and Lower Quartile Node Degree Noisy Disk Non parametric Deployment Figure 4.7: Closing the Node Degree Gap with non-parametric simulation yields similar node degree and variation. The distributions for non-parametric simulation and the empirical deployment are statistically equivalent.

80 through hybrid parametric/non-parametric models. 69

81 70 Chapter 5 Explaining the Prediction Gap In the previous chapter we saw that careful data collection in combination with nonparametric models can close the prediction gap left by certain parametric models like the Noisy Disk. This addresses the first two concerns listed in the introduction of Chapter 4: 1) deployments will be more predictable and 2) conclusions drawn from simulation-based analysis and comparison will be more meaningful. However, the third concern is still not addressed: the Prediction Gap is still difficult to explain. The techniques in Chapter 4 did not identify the aspects of our realworld environment and range sensor that caused the error not predicted by the Noisy Disk. Without knowing the cause of this error, it is still difficult to reduce it. Any aspect of our ranging data that does not conform to the traditional Noisy Disk model can be called a ranging irregularity. Typically, a real-world range sensor may have dozens of ranging irregularities due to manufacturing flaws, changing environments, or unforeseen physical dynamics. In this section, we develop a scientific approach to identify the irregularities in our empirical ranging data that contribute to increased localization error. This analysis is complicated by the fact that all localization algorithms may react differently to ranging irregularities; an irregularity may cause significant error for one algorithm, not affect another algorithm at all, and even improve the error results for a third algorithm. To completely explore the ranging irregularities in our ranging data set, we formulate an experiment and repeat it with six different localization algorithms from the literature. The experimental setup allows us to consider a particular irregularity X in isolation, and to answer the question: Question Is ranging irregularity X a significant cause of error leading to the Prediction Gap? The experimental setup consists of two steps. First, we evaluate the algorithm with the Noisy Disk ranging model. Then, we add ranging irregularity X to our model and evaluate the

82 71 algorithm again. This experimental method isolates the effect of irregularity X and a comparison of the results from the two steps verifies one of two possible hypotheses: H 0 Irregularity X is not a significant cause of error, and localization error in both trials will be the same. H 1 Irregularity X is a significant cause of error, and localization error in both trials will be different. A key aspect of this methodology is that the experimenter does not need to know how or why irregularity X may affect the localization algorithm, and may not even have a clear idea of what irregularity X is. For example, the experimenter may want to test if the empirical noise distribution is different from the model, without knowing exactly how to characterize the difference between the two. We do not require the experimenter to modify the parametric form of the Noisy Disk model to try to capture irregularity X. Instead, we combine the Noisy Disk model with a non-parametric model of irregularity X. The techniques we use to add only a single irregularity at a time will be described in more detail in the next section. 5.1 The Experimental Setup The experiment we use to identify the cause of the Prediction Gap for a particular algorithm has four steps: 1. We compare the empirical ranging data to the Noisy Disk model and hypothesize which ranging irregularity is causing the Prediction Gap. 2. We develop a hybrid model that incorporates the ranging irregularity into the Noisy Disk model. 3. We derive Noisy Disk parameters from the empirical ranging data to ensure a fair comparison between the ideal and empirical components. 4. We evaluate the localization algorithm using both the hybrid model and the Noisy Disk model. By comparing the resulting localization errors, we can isolate the effects of the ranging irregularity. In Section 5.1.1, we compare our empirical ultrasound data to the Noisy Disk model and identify four ranging irregularities that may be causing the Prediction Gap. In Section 5.1.2,

83 72 we incorporate these irregularities into hybrid models. We derive Noisy Disk parameters from our empirical data in Section The results of performing this experiment on six different algorithms are presented in Section Identifying Ranging Irregularities We can hypothesize ranging irregularities through inspection of our empirical ultrasound data, illustrated in Figure 5.1. Figure 5.1.a shows that the probability of successfully obtaining a range estimate at each distance does not match the Unit Disk model of connectivity: many pairs that are closer than d max do not in fact obtain a ranging estimate with some probability while others farther than d max do. Figure 5.1.b contains a histogram of ranging error that illustrates non-gaussian ranging error, including a larger number of extreme underestimates and overestimates than would be predicted by the Normal distribution. Based on these observations, we hypothesize four types of ranging irregularities that might be causing the prediction gap: Extreme overestimates: an excess of range estimates that are longer than the true distance by more than two standard deviations Extreme underestimates: an excess of range estimates that are shorter than the true distance by more than two standard deviations Long-range proficiency: the existence of range estimates between nodes farther than nominal range d max Short-range deficiency: the existence of range failures between nodes closer than nominal range d max The causes of these irregularities are unknown, but may include irregular environmental attenuation, variance in node orientation, or irregular amplifying pathways Creating Hybrid Models We can isolate each of the four irregularities described above by creating a series of five different ranging models, each incorporating one ranging irregularity more than the previous one: Model 1 (Noisy Disk): No irregularities Model 2: Model 1 + Extreme overestimates

84 Probability of Connectivity Short Ranges Long Ranges True Dmax Frequency of Occurrence Underestimates Overestimates Distance (cm) Error (cm) Figure 5.1: Ranging Irregularities are evident from the empirical data, including (a) short-range deficiency and long-range proficiency and (b) extreme underestimates and overestimates. Model 3: Model 2 + Extreme underestimates Model 4: Model 3 + Long-range proficiency Model 5: Model 4 + Short-range deficiency After evaluating a localization algorithm with this series of models, the localization error produced by Models 1 and 2 can be compared to evaluate the effect of extreme overestimates. Similarly, the error produced by Models 4 and 5 can be compared to evaluate the effect of shortrange deficiency. Because we are adding each irregularity to a model with all previous irregularities, Model 5 will incorporate all four irregularities simultaneously. Thus, Model 1 is the pure Noisy Disk model and Model 5 is the pure empirical data. Models 2-4 are on the spectrum between these two extremes. By structuring our experiments in this way, we are able to observe the effects of each irregularity as well as their cumulative effects, and can compare the ideal model directly to the empirical model. However, we are also assuming that the effects of the different ranging irregularities are independent. A more complete study would present a comparison between all 2 4 combinations of ranging irregularities, although we have found through experimentation that the independence assumption is reasonable.

85 74 To create Model 1, we estimate ˆd ij according to the Noisy Disk formula listed in Equa- N(d ij,σ) d ij d max ˆd ij = (5.1) ø otherwise. tion 2.6 Model 3 contains both extreme overestimates and underestimates, i.e. Model 3 is empirical noise coupled with Unit Disk connectivity. To create this model, we sample an empirical value d from a new set of empirical observations M(d ij,ǫ), which includes only those that were not ranging failures M(d ij,ǫ) ø d ˆd M(d ij,ǫ) ij = ø d ij d max otherwise. (5.2) Model 2 is similar to Model 3 except that the range estimate ˆd should be normally distributed if it is less than zero and empirically distributed if it is greater than zero. We can achieve this distribution by replacing d with normally distributed noise whenever it is an underestimate d M(d ij,ǫ) d 0, d ij d max ˆd ij = N(d ij,σ) d < 0, d ij d max (5.3) ø otherwise. The noise in Model 4 is always distributed according to the empirical distribution, and range estimates are always obtained at distances less than d max. However, range estimates are also obtained at distances greater than d max with the same probability as empirical range estimates. To achieve this distribution, we use d ˆd M(d ij,ǫ) d ij d max ij = (5.4) d M(d ij,ǫ) d ij > d max Model 5 uses both the empirical noise and connectivity distributions, achieved with pure random sampling from the empirical data set ˆd ij = d M(d ij,ǫ) (5.5) Parameters and Topology Using these five models, we evaluate all six algorithms on a simulated 18m x 18m square topology with 100 nodes and four anchors, one in each corner. Nodes are placed in a grid topology

86 75 with Gaussian noise added to the grid positions to avoid exhibiting artifacts of the network partitions that are likely in a completely random topology or of the strict rigidity of a true grid, neither of which are representative of the canonical deployment. Regardless of the ranging model used, all networks have an average degree of 9, meaning that all nodes have an average of 9 neighbors. This number of neighbors has been shown to be a transition point between high and low density networks [50]. The only exception to this rule are trials with Model 4 from Section 5.1.2, for which it was impossible to hold both d max and average network degree constant. These trials have a higher network degree of 12 because the model incorporates long-range proficiency and all nodes closer than d max are connected. This experiment only measures a single point along the dimensions of network size, anchor density, topology, etc. The chosen topology is sufficiently representative of the canonical sensor field deployment, however, because smaller topologies exhibit predominantly edge effects (nearly all nodes are near an edge) and larger deployments can be subdivided into a network of this type by placing anchor nodes appropriately throughout the network. The purpose of this study is not to explore the effects of network size and anchor density, which has been done in other studies [50, 63, 81], but to explore the effects of ranging irregularities. In all of our experiments, the non-parametric models and Noisy Disk parameters σ and d max are produced from the ultrasound profile that was collected in our deployment environment, as described in Section 3. In order to explore the effects of outliers on these localization algorithms, we use mean filtering instead of the special mediantube filter that we designed to remove these outliers. To derive the parameter σ, we fit a mixture of Gaussians to the ultrasound data and choose the parameter σ with the highest likelihood, similar to Section 3.9. Unlike that section, which followed standard convention in the literature by setting d max to the maximum obtainable range of the sensor, we now set d max to a value that would achieve the same average degree as the empirical data. We call this value the effective range of the empirical data, which can be calculated as r eff = where P = r=dmax r=0 Π r 2 p(r) (5.6) and p(r) is the empirical probability of successfully obtaining a range estimate at distance r. Nodes using the Unit Disk model of connectivity with parameter d max = r eff should have the same number of neighbors as nodes using the empirical ultrasound data. P Π

87 76 Rank Ideal Empirical 1 DV-Distance MDS-Map MDS-Map 2 MDS-Map(P) Bounding Box DV-Distance 3 Bounding Box MDS-Map(P) Robust Quads 4 GPS-Free GPS-Free Robust Quads Figure 5.2: The Ordering of Localization Algorithms is not the same for the ideal ranging model and the empirical model. 5.2 Experimental Results We perform the experiment described above on six localization algorithms from the literature: Bounding Box, DV-distance, MDS-Map, GPS-Free, Robust Quads, and MDS-Map(P). These algorithms were chosen because they represent the two main classes of approximations used by multi-hop localization algorithms: the shortest-path and the patch and stitch approximations. The algorithms themselves were introduced in Section 2.4 and are more completely described in Section 5.3. The results of our experiment on these six algorithms reveal several broad findings: The Prediction Gap is evident with all six algorithms; no algorithm using empirical ranging data produced localization error within a factor of two of the Noisy Disk prediction. The cause of the Prediction Gap is different for each of the six algorithms; irregularities do not have the same effects on all algorithms. The ranking of the algorithms is different with the Noisy Disk model and empirical ranging data; an algorithm that appears to be better with the Noisy Disk model may actually be worse with empirical ranging data, or vice versa. The median errors and the median percentage of nodes localized in 30 trials of each algorithm with all five ranging models are shown in Figure 5.3. We used a one-tailed t-test with α = 0.05 to compare adjacent models, and the statistically significant changes are indicated in the figure with * s. For example, the Bounding Box and DV-Distance algorithms are both significantly affected by Models 3 and 4, but not by Models 2 and 5. Therefore, with these algorithms the difference between Models 2 and 3 and Models 3 and 4 are marked with a * while the difference between

88 * Model 1 Model 2 Model 3 Model 4 Model 5 Localization Error (cm) * * * * * * 200 * * 0 BBox DV D MDS GPS F RQds MDS(P) * (a) Localization Error * * Percent Nodes Localized (%) BBox DV D MDS GPS F RQds MDS(P) (b) Percent Localized Figure 5.3: Experimental Results for each algorithm along the x-axis, with each of five different ranging models, showing (a) the median localization error and (b) the median percentage of nodes localized. Statistically significant changes are indicated with * s.

89 78 Models 1 and 2 and Models 4 and 5 are not marked. The ranging irregularities that cause changes in each algorithm are summarized in Table 5.4. All algorithms perform relatively well when evaluated using the Noisy Disk model. The fact that localization error for all algorithms gets significantly worse as ranging irregularities are introduced, and that no algorithm improves, demonstrates that the Prediction Gap is a problem with all localization algorithms, not just the DV-distance algorithm as demonstrated in Chapter 3. Perhaps most surprising is the extremely high sensitivity of all six algorithms to small changes in the ranging model. Simulation with a theoretical model is never an exact replica of reality and is never expected to produce exactly the same algorithmic response as empirical noise. However, we do typically expect simulation to be 1. indicative: results should be within a constant fudge factor of empirical results 2. decisive: an algorithm that performs better in simulation should also perform better in reality The results of our experiment show that simulation with the Noisy Disk does not meet either of these expectations. Localization error increases by less than 70% for some algorithms and over 800% for others, indicating that localization error in simulation is not indicative of error in a real deployment. To test for decisiveness, we used a one-tailed t-test with α = 0.05 to derive a statistically significant ranking of all algorithms. An equivalent ranking between two algorithms indicates that they are both statistically better and worse than the same set of other algorithms, and that there is no difference between their own localization errors. The resultant orderings are not the same when using purely theoretical and purely empirical models, as shown in Table 5.2. This conclusively shows that simulation using the Noisy Disk ranging model is neither indicative nor decisive, meaning that is has almost no value when trying to design, tune, and deploy a localization algorithm in the real world. This result motivates the more detailed analysis presented in the next section that identifies what aspects of empirical noise are not captured by the Noisy Disk model, yet are having significant impact on localization simulations. Besides the trends mentioned above, the true value of these results is that we can explain the cause of the Prediction Gap for each algorithm. The numeric results and the significance testing illustrated in Figure 5.3 allows us to accept one of the two hypotheses above for each combination of ranging irregularity and localization algorithm. For example, we can conclude that the Bounding Box and DV-Distance algorithms are sensitive (H 1 ) to extreme underestimates and long-range proficiency (Models 3 and 4) because those models caused statistically significant changes in localization

90 79 Extreme Overestimates Extreme Underestimates BBox Error BBox % Localized DV-D Error DV-D % Localized MDS Error MDS % Localized GPS-F Error??? GPS-F % Localized???? RQds Error RQds % Localized MDS(P) Error MDS(P) % Localized Long-range Proficiency Short-range Deficiency Figure 5.4: Causes of the Prediction Gap for each algorithm are summarized here. For each column, and indicate that the ranging irregularity effects or does not effect the final localization error and the percentage of nodes localized by a particular algorithm. The? indicates that the experiment produced an inconclusive result.

91 80 (a) Consistent Bounds (b) Inconsistent Figure 5.5: The Bounding Box Algorithm (a) constrains each node to be within its multi-hop distance to each anchor (b) sometimes resulting in inconsistent constraints. error for those algorithms. On the other hand, these algorithms and not sensitive (H 0 ) to extreme overestimates and short-range deficiency (Models 2 and 5). Table 5.4 summarizes which ranging irregularities caused statistically significant changes in localization error and the percentage of nodes localized for each algorithm. Only two algorithms, MDS-Map and MDS-Map(P), show the same pattern of statistically significant changes in both localization error and the percentage of nodes localized. This indicates that, due to the different approximation algorithms and their correspondingly different assumptions, each algorithm is affected differently by each ranging irregularity. In the next section, we use our findings to analyze each approximation algorithm and to identify which assumptions they make that might not hold true with empirical ranging data. 5.3 Analyzing Each Algorithm Bounding Box The Bounding Box algorithm [82, 91] uses the shortest path distance sp ij to constrain the unknown coordinates of node i in terms of the known coordinates of anchor node j. These

92 81 (a) Normal (b) Noise (c) Range Figure 5.6: Shortest Paths (a) zig-zag and should always be longer than the true distance. However, (b) extreme underestimates and (c) long-range proficiency combine to significantly shorten them. constraints take the following form: x j e ij < x i y j e ij < y i < x j + e ij < y j + e ij These are very loose constraints which only require that a node be within a certain distance from an anchor node, not at a certain distance. Furthermore, the constraints are placed on the x and y coordinates independently, so the union of constraints from all anchor nodes defines a box, as depicted in Figure 5.5(a). The location of each node is then approximated to be the center of the box defined by the union of all constraints. The simulation results in Figure 5.3(a) show that localization error for the Bounding Box algorithm significantly increases when subject to extreme underestimates and long-range proficiency. Simultaneously, the percentage of nodes that the algorithm is able to localize drops significantly. However, the algorithm is largely impervious to extreme overestimates and short-range deficiency. We can explain these trends through a deeper analysis of how the shortest path approximation is affected by noise and connectivity irregularities. Intuitively, shortest path distances are always longer than the true distance because of their zig-zag nature, as illustrated in Figure 5.6(a). Shortest paths straighten out as the network density

93 82 increases and should asymptotically approach the true distance as density goes to infinity. However, any shortest path algorithm will preferentially choose underestimated ranging estimates and avoid overestimated range estimates, i.e. given a choice between two similar paths, the algorithm will choose the one with a higher proportion of underestimates because it will be shorter. This effect is illustrated in Figure 5.6(b). Therefore, the positive and negative errors in range estimates do not necessarily cancel out; shortest path algorithms are highly sensitive to underestimates and are largely impervious to overestimates. For this reason, the extreme underestimates in our empirical ranging data can cause disproportionate errors in shortest path estimates; they effectively create shortcuts through the network and divert many shortest paths, causing widespread errors through a wormhole effect. This is illustrated in Figure 5.7(a), which shows average shortest path distance errors for Models 1, 3, and 5. Shortest paths from Model 3, which contains noise irregularities, can be as much as 50% shorter on average than those from Model 1, even though the nominal empirical ranging error is only around 5 10%; the few range estimates that are extremely underestimated cause errors in a large number of shortest paths. The same graph shows that the shortest paths get even shorter with increased network density. This is because the distance vector algorithm has more reasonable alternatives to a shortest path in a very dense network than in a very sparse one and will therefore have more opportunities to use underestimates or avoid overestimates. The effects of underestimates are exacerbated by the effects of long-range proficiency. The shortest path algorithm prefers to use long ranges and largely ignores short ranges because long ranges tend to decrease the shortest path distance, as illustrated in Figure 5.6(c). Therefore, the fact that our empirical ultrasound data has more long ranges and less short ranges, i.e. that it has both long-range proficiency and short-range deficiency, means that shortest path distances will be decreased further, even though network density remains the same. This effect is illustrated in Figure 5.7(a) where, for each density, the shortest path distances are shorter for Model 5 than for Models 1 or 3. Thus, extreme underestimates and long-range proficiency combine to yield shortest path distances which are up to 50% shorter than the true distances. This is extremely detrimental to the Bounding Box algorithm because it can result in inconsistent bounds; when the shortest path distances become too short, the upper and lower bounds defined by two or more anchor nodes do not overlap. In this scenario, nodes cannot be localized, as seen in Figure 5.5(b). As noise and connectivity irregularities are introduced, the percentage of nodes localized by Bounding Box quickly drops to the extent that Bounding Box cannot localize most nodes when subject to Model

94 83 2 Model 1 Model 3 Model 5 Normalized Shortest Path Error Average Degree (a) Shortest Path Errors Model 1 Model 3 Model Percent Localized (%) Average Degree (b) Bounding Box: percentage localized Figure 5.7: The Effect of Density (a) is to make shortest paths get shorter as density increases and as ranging irregularities from Models 3 and 5 are introduced. In (b), this leads to many nodes not being localized by the Bounding Box algorithm.

95 84 5, even at low densities. The shortest path errors at each density are strongly correlated with the percentage of nodes localized by bounding box, as shown by the two graphs in Figure DV-Distance Like the previous algorithm, DV-Distance [63] approximates the distance between a node i and an anchor node j to be the shortest path distance sp ij. DV-Distance then uses this value to constrain the position of node i in terms of the position of each anchor node j with an equation of the following form: (x i x j ) 2 + (y i y j ) 2 = sp 2 ij In contrast to Bounding Box, this strict equality relates both x and y coordinates in the same equation, forming a circular constraint. A system of at least three such equations can be linearized and solved with least squares for the coordinates x i and y i, as explained in Section 3.5. In this way, DV-Distance directly solves for node position by using the shortest path distances to reduce the multi-hop localization problem to single-hop localization. Because DV-Distance makes the same shortest-path approximation as Bounding Box, it is susceptible to the same ranging irregularities: extreme underestimates and long-range proficiency. DV-distance solves directly for a point estimate of each node s position, however, and does not suffer from the problem of inconsistent bounds as Bounding Box does. Therefore, DV-Distance is always able to localize all non-anchor nodes in the network, as shown in Figure 5.3(b). To deal with noise in the shortest paths, DV-distance exploits the fact that each anchor node can calculate the ratio of the shortest path distance and the true distance to all other anchors. This ratio is broadcast to the network as a correction factor, which can be used by neighboring nodes to adjust shortest path estimates before localization. The correction factor at anchor k for anchor j would be of the following form: (xj x k ) 2 + (y j y k ) 2 C kj = (5.7) sp jk A node i near the anchor node k can improve its own shortest path estimate to j by multiplying it by the correction factor sp ij = C kj sp ij (5.8) Anchor correction factors are intended to fix exactly the kind of systematic errors in shortest path distances that trouble Bounding Box. Indeed, Figure 5.8 shows that corrections factors cause the median shortest path distance error with Model 5 to be very similar to that achieved with

96 Model 1 Model 5 Normalized Corrected Shortest Path Error High Variance Low Variance Average Degree Figure 5.8: Anchor Corrections reduce systematic bias, making the shortest path distances of Model 5 approximate those of Model 1. However, they do not remove the high variance that ranging irregularities cause in the shortest path algorithm. Model 1. However, the results in Figure 5.3(a) show that DV-Distance s localization error is affected by empirical ranging data almost as much as Bounding Box. Correction factors in fact are not very effective in the face of ranging irregularities because they do not remove the high variance in shortest path distances caused by Model 5. Figure 5.8 shows that corrected shortest path distances from Model 5 can be off by a factor of two both before and after correction factors. The reason for this high variance is that, as stated in Section 5.3.1, a small number of noise or connectivity irregularities cause errors in a large number of shortest path distances. However, they do not cause errors in all shortest paths, nor are all irregularities equally damaging. Therefore, some shortest path distances will be greatly affected by ranging irregularities while others will be unaffected. Anchor corrections apply to all shortest paths regardless, correcting any general bias in shortest path errors but not correcting the variance MDS-Map MDS is an analytical technique to derive n-dimensional positions of n objects given a complete similarity matrix D with the metric distances between them. MDS-Map [87] is a rangingbased sensor localization algorithm that uses MDS by making two approximations: 1) all range

97 86 failures e ij = ø can be approximated by shortest path distances sp ij to convert the incomplete matrix E into a complete similarity matrix D 2) the locations of all nodes can be approximated by a 2-dimensional projection of the n-dimensional positions derived through MDS. This procedure requires the entire graph G, so MDS-Map is a centralized algorithm. Even though MDS-Map also uses the shortest path approximation, results in Figure 5.3(a) show that it is much more robust to underestimated ranges than Bounding Box and DV-Distance. One shortcoming of the previous two algorithms is that they only use shortest paths sp ij between nodes and anchor nodes; DV-Distance uses one shortest path per anchor node, while Bounding Box only uses at most four shortest paths in total: those that define the highest and lowest bounds on its x and y coordinates. In contrast, MDS-Map uses edges between all nodes simultaneously, dramatically increasing the number of constraints used to determine a node s location and reducing the ability of a few underestimates to have significant influence. MDS-Map also shows a marked increase in sensitivity to extreme overestimates with respect the Bounding Box and DV-distance, which were both impervious to them. This is because Bounding Box and DV-Distance use shortest paths to estimate primarily long distances between nodes and anchor nodes while MDS-Map estimates the shortest path differences between all pairs of nodes, most of which are relatively close together. Shortest paths of one or two hops are more likely to be affected by extreme overestimates than shortest paths of many hops; the shortest path algorithm can usually choose between many alternative routes for long paths but not for short ones. The first two algorithms therefore showed a general bias towards underestimated shortest paths while MDS-Map shows instead a much higher total variance, with many underestimated and many overestimated edges GPS-Free While the previous three algorithms are what we call shortest path approximations, the next three are what we call patch and stitch approximations. The patch localization algorithm used by GPS-Free uses the center node i of the patch and two connected neighbor nodes { j,k e ij,e ik,e jk E} to bootstrap a coordinate system by assigning the following coordinates: (x i,y i ) = (0,0) (x j,y j ) = ( ˆd ij,0) (x k,y k ) = ( ˆd ik cos(γ), ˆd ik sin(γ))

98 87 where γ = arccos( ( ˆd ij ) 2 + ( ˆd ik ) 2 ( ˆd jk ) 2 2( ˆd ij ˆd ) (5.9) ik ) These three nodes become anchor nodes that define a local coordinate system. The other nodes in the patch are localized using iterative multi-lateration [82]: any node connected to at least three anchors is first localized and then its new position estimate is used to localize other nodes. This process repeats until all possible nodes are localized. GPS-Free chooses the two bootstrapped anchors j and k to maximize the total number of neighbors in the patch that can be localized through iterative localization. This criteria does not uniquely specify the pair, and in our implementation we randomly chose any pair that met this criteria. The only significant change to the GPS-Free localization error is caused by extreme overestimates. However, this algorithm is most likely sensitive to several ranging irregularities, but the effect is not statistically significant because the magnitude of the error has already approached that of random node placement. Therefore, we indicated most results of this study as inconclusive in Table 5.4. The extreme sensitivity of this algorithm to noise can be attributed to Iterative Multi- Lateration, GPS-Free s patch localization algorithm, which exhibits the same trend when subject to ranging irregularities. Iterative Multi-Lateration is highly sensitive to noise irregularities because each step in the process uses only a few range estimates, and subsequent steps build upon the results of earlier steps. There are very few extremely underestimated or overestimated ranges in the data set, but if one of them is used in an early stage of localization, the errors it causes will propagate to all other nodes in the cluster. The more surprising fact about GPS-Free localization is that the percentage of localized nodes is actually improved when subjected to long-range proficiency and short-range deficiency. These irregularities do not change the average degree of the network or the average number of nodes localized in each cluster. Instead, it makes the neighbors in each cluster slightly farther away from each other on average. This decreases the number of times that the intersection of two clusters N ij is a co-linear set, causing a slight increase in the number of patches that can be stitched Robust Quads Robust Quads is a patch and stitch approximation that attempts to improve on GPS-Free by preventing some of the biggest errors in iterative multi-lateration. The robust quads algorithm defines a parameter of robustness θ which is usually set to 3σ, where σ is the nominal standard deviation of ranging noise. A triangle of neighbors i,j, and k is defined to be robust if dcos 2 (φ) > θ,

99 88 Figure 5.9: Robust Quads with this cluster cannot localize the top six and the bottom three nodes in this cluster to a common coordinate system. At only 66% localized, this patch has a high probability of stopping the stitching process. where d is the shortest edge and φ is the smallest angle in ijk. A clique of four nodes i,j,k,l is defined to be a robust quad if all triangles between these nodes form robust triangles. To localize a patch, this algorithm first finds all robust quads from the ( N i ) 4 quadrilaterals in the patch. It forms an overlap graph G o = (V o,e o ) where each vertex q V o is a robust quad and vertices are connected iff the two quads have at least three nodes in common, i.e. e qp E o q p = 3, q,p V o. Three nodes from one robust quad are used to bootstrap a coordinate system, as in GPS-Free, and to localize the fourth node in that quad. Then, all other nodes in the patch are localized using iterative multi-lateration, with the order in which nodes are localized defined by a breadth-first search through the overlap graph. Because quads are fully connected, a quad that overlaps with a localized quad is guaranteed to have at most one unlocalized node that is connected to three localized nodes. Robust Quads does not specify how to choose the root of the breadth-first search. In our implementation, we choose any quad that contains the center node i of the patch. As with GPS-Free, we use the stitching order defined by MDS-Map(P). Because Robust Quads tends to localize nodes using long ranges and avoids localizing nodes using short ranges, it is very sensitive to extreme overestimates and relatively robust to extreme underestimates. The dominant characteristic of Robust Quads, however, is that with Models 1 3 less than 25% of the nodes can be localized to a global coordinate system in our topology. This is not true for the Robust Quads patch localization algorithm, which localizes on average 65% and 55%, respectively. Therefore, we can assume that the reason most nodes cannot be localized is

100 Model 1 Model 3 Model 5 Percent Localized (%) Average Degree Figure 5.10: Robust Quads goes very quickly from localizing almost no nodes to localizing almost all nodes. Long-range proficiency triggers this phase transition at lower densities than Models 1 or 3. because of a failure in the stitching algorithm. Through inspection of several networks in which no more patches could be stitched together, we found that many of the patches on the fringe of the localized section were similar to the patch illustrated in Figure 5.9. This patch contains five robust quads, the top three of which overlap and the bottom two of which do not. Therefore, the top six nodes can be localized to a common coordinate system while the bottom three can not, i.e. 66% of the nodes could be localized. This causes a problem during the stitching process when only half of the cluster overlaps with a cluster that is localized to the global coordinate system; the cluster cannot propagate the coordinate system to the other side of the cluster, and stitching stops. In a network where most patches are localized to only 50 60%, the probability of each patch stopping the stitching process is high enough that only a small number of patches can be stitched. Subjecting Robust Quads to long-range proficiency improves the percentage of nodes that can be localized to a global coordinate system. In the patch localization process, the percentage of nodes localized increases to about 80%. This is because a patch with more long ranges is more likely to have robust triangles. This is similar to the reason why GPS-Free localized more nodes

101 90 (a) Not a Robust Quad, but robust (b) Not a quad, but robust Figure 5.11: Robust Quads is Overly-restrictive in the sense that it will not allow the three anchor nodes in either of these topologies to localize the fourth node, even though it could be performed robustly. when subjected to long-range proficiency. While a change from 50 60% to 80% is not very high, it is enough to trigger a phase transition in which the probability of each patch stopping the stitching process becomes low enough that most patches can be stitched. This phase transition is evident in Figure With both Models 1 and 3, the transition occurs between average degrees of 10 and 12. When long-range proficiency is introduced, however, the transition occurs between degrees of 6 and 7.5. Long-range proficiency also causes the average localization error to increase, approaching that of GPS-Free. This is presumably because the new nodes that are being localized are those with higher localization error. It is worth mentioning that the entire patch in Figure 5.9 can be localized with respect to a single coordinate system without any danger of flip or discontinuous flex ambiguities described by Moore [60]. The patch is not completely localized, however, because the Robust Quads patch localization algorithm is too restrictive in that it does not allow multi-lateration even in cases where it can be performed robustly, as illustrated in Figure 5.11.

102 MDS-Map(P) MDS-Map(P) uses MDS-Map as a patch localization algorithm. The shortest paths between all nodes in patch i are calculated and combined with the range estimates to form a complete similarity matrix D i, which is used to localize the nodes in the patch relative to each other. No anchor bootstrapping is required. The local coordinate systems are then stitched together to form a global coordinate system using the algorithm described earlier. The original MDS-Map(P) proposal suggests using patches of nodes that are more than one hop from the center of each patch. In our implementation, we use one-hop patches to make the algorithm comparable to the GPS-Free and Robust Quads algorithms. The characteristics of MDS-Map(P) follow the same trends as those of MDS-Map: the algorithm is more sensitive to connectivity irregularities than to noise irregularities, although significantly affected by both. The main difference between the two algorithms is that MDS-Map(P) has consistently higher error, regardless of the ranging model used, because the stitching algorithm is amplifying the errors introduced during the patch localization process. By comparing the localization of MDS-Map and MDS-Map(P) on a network that required chains of up to 25 stitches, we can infer that the stitching process amplifies errors by a factor of two or less. One problem with GPS-Free and Robust Quads is that they use a chain of calculations (i.e. iterative multi-lateration), each of which depend on a small number of ranging estimates. A single noisy ranging estimate that causes an error early on can be very damaging due to error propagation. MDS-Map(P) makes headway on this problem by replacing the fragile chain with a relatively robust single computation for patch localization. However, it does not do the same for the stitching algorithm. All algorithms including MDS-Map(P) use a greedy stitching algorithm that stitches the un-stitched patch that has the largest overlap with a stitched patch. This stitching algorithm is not robust to noise: a single badly localized patch can (and does) cause severe stitching error, which propagates through all subsequent stitches.

103 92 Chapter 6 Removing the Prediction Gap In this chapter, we close the Prediction Gap by building a parametric model of our raw ultrasound data. In the previous sections, we have seen that the traditional parametric model is insufficient and that non-parametric models provide much more accurate simulation. However, parametric models are preferable to non-parametric models for several tasks, including analytical proofs and the mathematical derivation of error bounds. The value of a parametric model in this regard, however, depends on its balance of accuracy and simplicity. Capturing all irregular features of the data may require a sophisticated model with a large number of parameters, which would be difficult to use for analysis. Simplifying the model and reducing the number of parameters, on the other hand, can reduce the accuracy of the model and make any such analysis meaningless. The goal of this chapter is to explore the tradeoff between accuracy and simplicity in the context of ultrasonic range sensors by creating a simple model that captures the most salient features of our empirical ultrasound data. The aspects of our data that we would like to capture are motivated by the hybrid model analysis in Chapter 5, which identified four irregularities in ranging connectivity and noise characteristics that can effect localization error. Some of these irregularities are only slight deviations from existing models. For example, even though ranging noise roughly conforms to the Normal distribution, a relatively small number of outliers can be devastating to some localization algorithms. This indicates that our model must capture not only the general structure of the data, but must also exhibit small deviations from that structure.

104 Existing Models of Irregularity The Unit Disk model of connectivity is known to be a simplification of true link-layer characteristics. Instead of only two regions of connectivity, where all nodes are completely connected at close distances and all nodes are completely disconnected at far distances, classical models would also predict a third region of unreliable connectivity where nodes are connected with some probability 0 < p < 1. This so called transitional region occurs when the signal to noise ratio (SNR) is high enough that packets can still be received but low enough that the probability of bit errors becomes substantial. This region is expected to be small, and classical models would predict that in conditions where a low-power radio has about 20m range, it might have a transitional region of about 2m [104]. However, several recent empirical studies have demonstrated that the transitional region can actually occupy over half of the maximum usable range [23,99,101], as illustrated in Figure 6.1. Effects of the transitional region can therefore dominate certain aspects of wireless networking, changing the expected pattern of network floods, the structure of routing trees, and the techniques used for reliable data collection. Because of its large impact, several new models have been created to explain the unexpectedly large size of the transitional region. In this section, we attempt to use several of these models to explain the ultrasound connectivity irregularities identified in previous sections. Short-range deficiency and long-range proficiency in ultrasound have many properties in common with the transitional region in radio links. One key difference is that this region occupies the entire useful range of ultrasound while it only occupies a portion of radio range, so any explanation to be borrowed from the wireless networking literature would need to be especially prominent in ultrasound. It would also need to be consistent with the many differences between the physical models of radios and range sensors Non-uniformity of Nodes One popular belief is that the transitional region in radio connectivity is caused by a nonuniformity of nodes; variations in radio circuitry, antenna orientation, etc. may cause some nodes to transmit with more power than others or to be more sensitive receivers, causing each node to have a different maximum range. Even if the individual nodes have a narrow transitional region, therefore, the transitional region for all nodes in aggregate would be wider than expected, as illustrated in Figure 6.2. A similar phenomenon could occur with ultrasonic range sensors: variance in the time constants of the oscillators or in the orientation of the reflective metal cone could cause some nodes

105 Figure 6.1: The Transitional Region for radio connectivity violates the Unit Disk model of connectivity and can impact several wireless networking protocols. With low-power radios, it can occupy over half of the usable range, as shown in this figure. Figure reproduced with permission [99]. 94

106 95 (a) Narrow Transitional Region for Individual Nodes (b) Wide Aggregate Transitional Region Figure 6.2: Non-Uniformity of Nodes is believed to cause a wider than expected transitional region; if a) each node has a narrow transitional region individually, but different range then b) the transitional region in the aggregate would appear wide. to transmit louder or receive with a lower SNR than others, causing each node to have a different range. To test this theory, we analyzed the probability that each node would obtain a range estimate in each of five different regions: 1-100cm cm cm cm cm The data used for this analysis is the same data collected in Section that is shown in Figure 3.2. The data collection process required that each node range to all other 24 nodes at up to five different distances. A high variance among nodes would be observed if some nodes could range only at short distances, other nodes could range at both short and mid-distances, and the rest could range at all distances. Figure 6.3, however, shows that our ultrasound range sensors do not exhibit this trend. There is some variation among nodes, but the transitional region for each individual node is comparable to the aggregate transitional region for all nodes. All nodes are almost equally likely to

107 96 obtain a range estimates in the five different regions. For clarity, this figure only shows data for 12 random nodes of the 25 used in the data collection process. Our analysis of ranging errors in Section 3.3 also supports the conclusion that there is low variance among the individual ultrasound devices. In that section, we calibrated the nodes using a single set of calibration parameters for all nodes, and then calibrated them again using different calibration parameters for each node to account for any variance among them. If there was a high variance among devices, the second calibration process would produce much lower range errors. However, we saw that the two calibration methods produced very similar range errors, indicating that there is little variation among nodes Radio Irregularity Model Another explanation in the wireless networking literature for the unexpectedly large transitional region in low-power radio links is called the Radio Irregularity Model (RIM) [102], which argues that the radio range is non-isotropic; instead of having a single range in all directions, the range boundary varies based on the angle of transmission, as shown in Figure 6.4. According to this model, a node would only be connected to a portion of the neighbors between the outer and inner limits of the range boundary, which would create the illusion of a transitional region. Non-isotropic propagation could be caused by shadowing and multi-path effects, as well as attributes of the radio itself. Especially with platforms like the mica mote, the layout of the hardware components with respect to the radio and antenna could effect the propagation of low-power radio signals. A similar phenomenon could feasibly be observed with our ultrasound transducers; if the reflective cone were off-center, as shown in Figure 6.5, the acoustic signal would be stronger in one direction than another. To test the RIM model against our ultrasound data, we analyzed the pattern of connectivity for each of our nodes in the two-dimensional topology shown in Figure 4.3(a). The RIM model would predict that, once a node fails to range to one node, it will fail to all other nodes at the same angle of transmission. However, the contour map in Figure 6.6, which illustrates the probability with which a node can obtain a range estimate to a neighbor, shows that the probability of ranging does not decrease monotonically: some nodes yield a higher probability of obtaining a range estimate than closer nodes that are at the same angle of transmission. A weak reflection of acoustic energy in a particular direction would affect all nodes in that direction, and so isolated spatial holes in connectivity violate the predictions of the RIM model.

108 cm cm cm cm cm Probability of Reception Node ID (a) 12 Nodes as Transmitters cm cm cm cm cm Probability of Reception Node ID (b) 12 Nodes as Receivers Figure 6.3: Uniformity of Ultrasound Nodes This graph shows the probability that a node can obtain a range estimate at each of 5 different distances, when acting as a) the transmitter and b) the receiver. All nodes are equally likely to obtain range estimates in each of the five regions, showing that non-uniformity is the not the cause of the wide transitional region in ultrasound.

109 Figure 6.4: The Radio Irregularity Model postulates that transmitters have non-isotropic range which varies with the angle of transmission. This could cause the illusion of a wide transition region because nodes at the same distance could be either connected or disconnected. Figure reproduced from [102] with permission. 98

110 99 Figure 6.5: Sources of Non-isotropic Ultrasound could include, for example, an off-center reflective cone, as shown in this figure. This would cause a non-isotropic transmission range, as predicted by the RIM model Gaussian Packet Reception Rate Model Because the cause of the transitional region is not obvious, early work in reliable multihop routing with low-power radios attempted to build a statistical model of the transitional region, without explaining the phenomenon [99]. At each distance, the model used the mean and standard deviation of the packet reception rates for all pairs of nodes observed at that distance. This model applied to the data in Figure 6.1 can be illustrated by the mean and error bars in Figure 6.7. This model does not explain the transitional region and does not necessarily provide an accurate description. In fact, it yields probabilities outside the range [0, 1]. However, the model described the transitional region in radio links well enough to be useful for reliable multi-hop routing, and may similarly be useful for multi-hop localization with ultrasound. Applying the model to our ultrasound data, however, reveals that it does not capture the bi-modality of the connectivity data; most nodes have either a very high or very low probability of obtaining a range estimate, but the Gaussian assumption of this model can only capture one or the other, as shown in Figure 6.8. At far and near distances, the model places most probability density of unconnected and well connected pairs, respectively. At mid distances, the model places most probability density in between, where only a small percentage of nodes can be found.

111 Figure 6.6: Ultrasound Connectivity Contours show that connectivity is non-monotonically decreasing: close nodes will often be disconnected while far nodes are connected, even when angle of transmission is held constant. This is not explained by the RIM model. 100

112 101 Figure 6.7: The Gaussian Packet Reception Rate Model uses aggregate statistics like the mean and variance of packet reception rates at each distance to model the transition region. Figure modified from [99] with permission.

113 Probability of Ranging Distance (cm) Figure 6.8: Ultrasound Connectivity cannot be modeled with an average and standard deviation at each distance because the model does not capture the bi-modal distribution observed at most distances.

114 103 Figure 6.9: Shadowing and Multi-path can combine to create a wide transitional region. Each pair of nodes will observe different signal attenuation due to shadowing and multi-path, causing them to approach the noise floor at very different distances. Figure reproduced with permission [104] Shadowing and Multi-path The last model we examine explains the unexpectedly wide transitional region in terms of natural variation in signal attenuation [104]. The radio signal is assumed to attenuate according to a log-normal path loss model given by PL(d) = PL(d 0 ) + 10n log 10 ( d ) + X σ (6.1) d 0 where d 0 is a reference distance, n is the coefficient of attenuation, and X σ is zero-mean Gaussian noise. Zuniga et al. assume that X σ is a constant value for a particular transmitter/receiver pair due to effects like shadowing and multi-path, 1 and that the Normal distribution only hold in the aggregate over all pairs [104]. Thus, a particular transmitter/receiver pair will consistently receive a stronger or weaker signal than the path loss model would predict, bounded with high probability by PL(d) ± 2σ 1 Shadowing is signal attenuation due to an obstacle between the transmitter and receiver and multi-path is signal energy received from reflective surfaces.

115 104. The probability of a bit error while receiving a radio message is given by P e = 1 2 exp( SNR ) (6.2) 2 This value is approximately 0 when the SNR is high and exponentially approaches 1 2 as SNR goes to zero. If the transitional region is defined to be the distance where the packet reception rate is 0.1 < p < 0.9, the beginning d s and end d e of the transitional region can be derived as d s d e = 10 Pn+10 log 10 = 10 Pn+10 log 10 1 ( 1.28 ln(2( f ))) Pt +PL(do)+Xσ 10n (6.3) 1 ( 1.28 ln(2( f ))) Pt +PL(do)+Xσ 10n (6.4) where f is the number of bytes in a packet. The value of X σ for a particular transmitter/receiver pair can greatly change the position of the transitional region, as shown in Figure 6.9. In this scenario, when X σ = 0 the transitional region occurs between 18 20m. When X σ = 2σ, it occurs between 11 12m, and when X σ = +2σ it occurs between 29 32m. Thus, according to this model, the cause of the transitional region is not inherent in the nodes themselves or in the angle of transmission, but in the two-dimensional space between the transmitter and receiver that causes a particular combination of shadowing and multi-path effects. Non-isotropic transmission and reception, as described by the RIM model above, may contribute to this effect. The concepts behind this model may explain the connectivity characteristics in question, but the physical model of ultrasound ranging is very different from the model of radio communication. For example, the probability of an ultrasound error does not approach 1 2 as SNR goes to zero, and the probability of successful transmission does not decrease exponentially as the packet length grows. Quite the opposite, the longer an ultrasound transmission, the higher is the probability that it might be received. Furthermore, any model of ultrasound ranging must not only explain connectivity characteristics but also the observed noise distribution. The following section adapts several of the underlying concepts from this model to the physical dimensions of ultrasound ranging. 6.2 Towards a New Parametric Model In this section, we attempt to formulate a parametric model of ultrasonic ranging. Unlike the models of the transitional region in radio connectivity described above, this model must capture both connectivity and noise characteristics. As with the radio models, we will assume log-normal

116 Figure 6.10: The Ultrasonic Emanation Pattern is the result of reflecting a cone with 25 degree spread into a plane. The acoustic energy is distributed on the frontal surface of the resulting radially distributed cone, with surface area of 4πd 2 tan( 25o 4 ). 105

117 106 signal attenuation, as given by Equation 6.1. The coefficient of attenuation n can be derived from a physical model of ultrasound propagation. The ultrasonic transducers used in our implementation emanate acoustic energy in a cone with a 25 degree spread. The 45 degree metal reflective surfaces placed above the transducer reflect that energy into a 360 degree, radially distributed cone with a 12.5 degree vertical spread, as illustrated in Figure At distance d, the outer edge of this region has height h = 2dtan( 25o 4 ) and width w = 2πd. The acoustic energy is therefore distributed over a surface with area A = h w = 4πd 2 tan( 25o 4 ). Thus, the density of acoustic energy should decrease proportional to 1, i.e. the theoretical coefficient of attenuation for our ultrasonic transducers is 2. d A Geometric Noise Distribution Once the acoustic energy arrives at the receiver, it will detect the first full wavelength of the signal with probability p detection. If we allow the startup time of the oscillator to be accounted for by the calibration process, we can make the simplifying assumption that every successive full wavelength of the signal will also be detected with the same probability p detection. The number of wavelengths ω needed to first detect an ultrasound signal will then follow the geometric distribution, given by P(ω) = (1 p detection ) ω p detection (6.5) We can assume the probability p detection is related to the signal attenuation PL(d). Unlike the previous model, however, the probability does not approach 1 2 as the received power approaches zero. Instead, the probability of detecting an ultrasound signal approaches zero as attenuation increases. This relationship can be given as p detection = exp( PL(d) α ) (6.6) where α is a scaling factor that determines the rate of attenuation over distance. The model is actually slightly more complicated because all received signals above a certain power rail the amplifiers in our ultrasound circuit and therefore have approximately the same probability of being detected. If P t is the transmission power then the power received at a certain distance is P r = P t PL(d). This value is lower than the maximum receivable power P rail by the quantity BR(d) = max(0,p rail P r ), and the probability with which the signal is detected becomes p detection = exp( BR(d) α ) (6.7) Before the ultrasound signal arrives and after it has ceased, we can assume that a node will generate false positives with a constant probability p f, i.e. the node will incorrectly detect a

118 107 signal when none is present. The probability of detection can be given for each of three different regions: 1) before the ultrasound arrives 2) while the ultrasound is being received, and 3) after the ultrasound pulse has ceased. Assuming v is the speed of sound, f is the ultrasound frequency, and τ is the time duration of the ultrasound pulse, these probabilities are given by p before (ω,d) = (1 p f ) ω p f (6.8) p during (ω,d) = (1 p f ) d f v (1 exp( BR(d) d f )ω α exp( BR(d) α ) (6.9) p after (ω,d) = (1 p f ) ω τ f (1 exp( BR(d) α ) τ f p f (6.10) With our ultrasound implementation, f = 25000, τ = 0.008sec, and P t = 75dB. The transducers have a maximum sensitivity of 60dB. We used our model with parameters σ = 17dB and α = 10 to generate data at the same distances used to collect the empirical data in Figure 3.2. The generated data, shown in Figure 6.11(a), captures several salient features of the empirical data set, which is reproduced in Figure 6.11(b) for clarity at short distances, ultrasound is detected quickly the nominal time to detect ultrasound increases with distance only a small number of detections require more than 4ms at all distances false positives are uniformly distributed both before and after ultrasound arrives false positives are more dense before ultrasound arrives than after An Exponential Model of Connectivity The geometric distribution appears to capture the speed with which an ultrasound signal is detected, and therefore accurately captures time of flight noise characteristics. However, it does not accurately capture connectivity characteristics. Because we used 8-millisecond ultrasound pulses with a signal period was 40-microsecond, the geometric model would predict that a signal remains completely undetected with probability p received = (1 exp( BR(d) α )) (6.11) Unfortunately, this is an overestimate. Figure 6.12 shows a kernel regression of the frequency with which each pair obtains a range estimate at each distance. In the empirical data set, this frequency

119 Time of Flight (msec) True Distance (cm) (a) Data Generated from Geometric Noise Distribution Time of Flight (msec) True Distance (cm) (b) Empirical Data Figure 6.11: The Geometric Noise Model produces data shown in a). This data captures many of the salient features of the empirical ultrasound data, shown in b).

120 109 follows a binary distribution: most pairs at short distances yield 100% while a small number yield 0% of range estimates. At longer distances, an increasing number yield 0%. In the generated data, however, almost all pairs yield between %. Connectivity is certainly related to signal attenuation P L(d), but the random component causing the connectivity characteristics are orthogonal to that causing the noise characteristics. This is evident from the empirical data at short distances, for example, where almost 20% of range estimates are not received at all, but those that are received are detected very quickly. This pattern is possible if connectivity can be determined by a binary indicator, such as an off-center reflective cone for example, but, given that the cone is not off-center, the speed with which a signal is detected is affected only by environmental factors. For this reason, we use independent random noise components X σ 1 and X σ 2 in the equations for deriving noise and connectivity components, respectively. In our ultrasound ranging hardware, we used an analog comparator to detect ultrasound signals above a certain power threshold P thresh. A more complete model of connectivity would therefore predict that yield decreases exponentially as the received power approaches P thresh. The amount that a received signal is above threshold is AT(d) = max(0,p r P thresh ), providing the following probability of detecting a signal p received = 1 exp( AT(d) α ) (6.12) The values p received and p detection are similar, but are dependant on inversely proportional factors; the amount that P r is below the rail BR(d) increases with distance while the amount that it is above threshold AT(d) decreases. The probability with which a signal is detected p detection decreases slowly as P r decreases. When P r approaches P thresh, however, the signal can no longer be detected and p received drops very quickly. The different power relationships and their significance with respect to p detection and p received are depicted in Figure We can apply this exponentially decreasing model of connectivity to the geometric noise model in the previous section by first calculating p received and generating a random value r = [0,1]. We can then replace Equation 6.7 with the following equation exp( BR(d) α p detection = ) r < p received otherwise p f (6.13) Thus, if the acoustic signal is detectable, it will be detected with probability p detection. Otherwise, it will be detected with the same probability as a false positive.

121 110 x 10 3 Probability Density (kernel smoothing) Ranging Yield Distance (cm) (a) Yield of Geometric Noise Model x 10 3 Probability Density (kernel smoothing) Ranging Yield Distance (cm) (b) Yield of Empirical Ultrasound Figure 6.12: Connectivity Characteristics of the Geometric Noise Model are very different from those of the empirical ranging data.

122 Figure 6.13: Power Relations illustrated in this diagram are used for modeling ultrasound. P t is the transmission power, PL(d) is the path loss at distance d, P r is the received power,and P rail and P thresh are the maximum and minimum receivable powers, respectively. p received is proportional to BR(d) = P rail P r while p detection is proportional to AT(d) = P r P thresh. 111

123 Time of Flight (msec) True Distance (cm) (a) Data Generated from Complete Model x 10 3 Probability Density (kernel smoothing) Ranging Yield Distance (cm) (b) Connectivity Characteristics of Complete Model Figure 6.14: The Complete Parametric Model combines the Geometric noise distribution with the exponential model of connectivity. This model produces a) noise characteristics and b) connectivity characteristics that represent the main features of the empirical ranging data.

124 113 We used the complete model to generate data at the same distances used to collect the empirical data in Figure 3.2. The generated data, shown in Figure 6.14, captures empirical connectivity characteristics as well as our simple geometric model of noise. However, the connectivity characteristics of the generated data now also match those of the empirical data set. Instead of most range estimates being received between % of the time, as they were in Figure 6.12(a), most estimates are received with probabilities of either 0 or 100% Verifying the Model The previous two sections provide a bottom-up approach to modeling, which focuses primarily on inspection and comparison of the salient features in the data set. However, inspection is not a complete metric of evaluation for a model because it relies on our understanding of the data set s salient features. In this section, we provide a top-down model verification process in which we run a localization algorithm on data generated from this model and compare the resulting localization error with that observed in a real-world deployment. The results of our comparison are shown in Figure 6.15, which shows the localization error in our empirical deployment and compares with the error predicted by the Noisy Disk model, the non-parametric model, and our new parametric model. Based on the fact that our new parametric model predicts the empirical deployment results as well as the non-parametric model, we conclude that this is a sufficiently accurate model of ultrasound ranging.

125 Upper Quartile, Median, and Lower Quartile 200 Localization Error (cm) Noisy Disk Non parametric Generated Deployment Figure 6.15: A Top-down Parametric Model Evaluation is possible by using data generated from the model in simulation and comparing with an empirical deployment. This figure shows that the model predicts the empirical deployment as well as the non-parametric model.

126 115 Chapter 7 Conclusions The Prediction Gap is a crippling problem in localization for three main reason: 1. The localization error of a real deployment is difficult to predict. This can be a problem for mission critical deployments which can only be deployed once, such as forest fire tracking, or for large deployments with 1000 s of nodes where the cost of redeployment is prohibitive. 2. Localization algorithms are difficult to evaluate and improve. Improvements made to an algorithm in simulation do not necessarily translate to improvements in a real deployment because the sources of error may be very different. Similarly, a comparison of two algorithms in simulation may not produce the same conclusion as a comparison on a real deployment. 3. Empirical error is difficult to explain. If everything known about the environment and range sensor is incorporated into a theoretical model which produces low errors in simulation, then the cause of any additional error observed in the real deployment is not known. If the cause of the additional error is not known, it is difficult to reduce. In this study, we thoroughly address the problem of the Prediction Gap. We first establish the existence and magnitude of the Prediction Gap by building and deploying a sensor localization system and comparing observed localization error with predictions from the traditional model of ranging. We then develop new non-parametric modeling techniques that can use empirical ranging data to predict localization error in a real deployment. These non-parametric models require a special form of data collection that ensures a thorough, high-resolution profile of our range sensor and its environment. That empirical profile is then used directly in simulation through statistical

127 116 sampling techniques. Our non-parametric models are key to closing the Prediction Gap, and solve many of the issues present in existing simulations. Once we close the Prediction gap, we proceed to identify its causes by creating hybrid models that combine components of our non-parametric models with traditional parametric models. By comparing localization error from a hybrid model with a purely parametric model, we isolate the effects of a single component of our data. We use this technique to identify the causes of the Prediction Gap for six different localization algorithms from the literature. We then use this knowledge to develop a new parametric model that captures the true characteristics of our empirical ranging data. 7.1 Advantages of Our Modeling Techniques Traditional techniques for the creation and validation of a sensor model in order to simulate a deployment can be as difficult as performing the deployment itself. One must first collect data in a sufficiently representative set of sensor contexts and distill this data down to a simple algebraic form. This new model must then be validated by comparing its predictions in simulation to a real deployment, and if the application behavior changes at scale or with different network topologies, the model may need to be validated with several different large networks. Finally, if the model fails the validation process, the scientist must debug the model by manually comparing the algebraic form with the raw sensor data in order to identify discrepancies. If any of the algorithm, sensor, stimulus, or environment change, the validation process must be repeated. Failure to validate a model may result in a discrepancy between simulation predictions and real-world deployments, which we call the Prediction Gap. The techniques we have demonstrated in this study greatly simplify the process of modeling a sensor. First, we demonstrated that a complete empirical profile of a sensor can quickly and efficiently be collected by using a large number of sensors simultaneously in the correct configuration. Second, if the data will only be used for simulation, the data does not need to be analyzed and distilled into an algebraic form; the data can be used directly in simulation through non-parametric modeling techniques. Finally, if a parametric model is needed, hybrid models allow us to refine a simple parametric model by systematically identifying discrepancies between it and a non-parametric model. Hybrid models allow us to remove one discrepancy at a time, in contrast to the standard validation process which requires all discrepancies to be removed at once. This methodology combines aspects of bottom-up modeling, in which the user analyzes the raw data and

128 117 makes a structural conjecture, and top-down modeling, in which the user validates that conjecture based on a particular usage. The advantages of non-parametric models go beyond not needing to postulate an algebraic form; they also make fewer assumptions that need to be validated. Validation of a parametric model verifies that 1) the structural features captured by the model accurately represent the data from a particular range sensor in a particular environment and 2) the structural features that are not captured by the model are inconsequential for a particular algorithm. Validation of our non-parametric model, on the other hand, verifies only that all the data in the set M(d,ǫ) is similarly distributed; if data from some nodes were very different than data from other nodes, for example, the non-parametric model we used would not accurately reflect the true deployment. Because the non-parametric model does not make assumptions about the structure of the data, it does not need to be revalidated when the range sensor or environment changes, as long as the data in M(d,ǫ) continues to be similarly distributed. Because the non-parametric model does not eliminate inconsequential features from the data, it also does not need to be re-validated when used with a different algorithm that may be sensitive to these features. 7.2 Parametric vs. Non-parametric Models As described above, our new non-parametric modeling and hybrid modeling techniques are crucial in explaining the Prediction Gap. However, non-parametric models are not the only way to remove the Prediction Gap. In this study, we demonstrated two techniques to achieve this: 1) to replace parametric models with non-parametric models in simulation, and 2) to make a more realistic parametric model. Each of these approaches has its advantages and disadvantages, which we explore here. One main benefit of parametric models like the Noisy Disk is that they provide insight to the algorithm designer by identifying only a small set of ranging characteristics; there may be hundreds of aspects of the physical world that affect each range estimate, but the model indicates that only a few of those features are actually important to localization algorithms. Furthermore, the algebraic form of the model can be useful for theoretical analysis. These strengths of parametric models, however, are also their weakness; parametric models need to be reevaluated and redeveloped for every new range sensor and environment. This is a tedious process requiring data collection and careful analysis followed by a model verification process that, to be complete, would require real localization deployments. Another problem is that empirical ranging characteristics like those

129 118 shown in Figures 4.1 can be too complex to capture in parametric form without some amount of simplification. Non-parametric models solve both of these problems: new models do not need to be created for new empirical distributions and complex ranging characteristics can easily be captured. However, non-parametric models do not provide an algebraic form that can be used for theoretical analysis nor do they provide any insight into the data or the algorithm. For example, although our simulations in Section 4.3 closed the prediction gap of our empirical deployment, it did not indicate what caused the error in the real deployment, or why the non-parametric model was a better predictor than the Noisy Disk model. In practice, the use of parametric and non-parametric modeling carry similar costs in all respects. Both require vast data collection that can be performed in a comparable amount of time. During simulation, both methods require a single random number to be generated for each range estimate. The Noisy Disk model requires the user to estimate parameters σ and d max from the data while our non-parametric model requires the user to choose a value ǫ. Perhaps the biggest cost of parametric models is that the user must choose an algebraic form. However, our non-parametric technique requires the user to properly bin the empirical data into subsets that contain similar data. We chose to bin our data as M(δ,ǫ), for example, although other bins might be more appropriate. For example, variations between radios and antennas can be modeled by parameterizing each node with the quality of its transmitter and receiver. These parameters can be estimated from empirical data using joint calibration techniques [95]. During simulation, each radio could be randomly assigned transmitter and receiver parameters T and R and data could be pooled and drawn from subsets M(δ,ǫ,T,R). As long as the parameters T and R are assigned according to the true distribution of radios, this should more accurately model non-uniformity of nodes than using subsets M(δ,ǫ). 7.3 Extending Analysis to Other Areas The area of ranging-based sensor localization is particularly vulnerable to the Prediction Gap and was chosen for this study because it has the following properties: It relies on a physical sensor that can be noisy, irregular, and easily influenced by the environment. In contrast to sensor systems that perform data collection, a localization algorithm must ac-

130 119 tually process the sensor data. As such, any theoretical model must accurately represent the physical sensor. Thorough evaluation of a localization system is necessarily performed in simulation because performing hundreds of large-scale deployments in different topologies is impractical. Therefore, theoretical models are necessary for research in localization. Small deployments do not necessarily demonstrate the Prediction Gap, and large deployments are rare. Unlike the area of wireless networking, where years of research at the physical, link, and MAC layers helped build realistic models of the radio that could be used for research at the routing layer, localization has not had sufficient reality checks for the most commonly used theoretical models of ranging. These properties, however, are not unique to ranging-based localization. They are also true for many other distributed sensing algorithms, such as tracking. In August 2005, we deployed 557 nodes with passive infrared (PIR) sensors in a field covering approximately 50,000 square meters [18]. The sensors themselves are shown in Figure 7.1(a), and part of the deployment area is shown in Figure 7.1(b). The Markov Chain Monte Carlo Data Association (MCMCDA) algorithm was used on the output of sensors to track multiple objects moving through the field [64]. This algorithm considers data from multiple parts of the network simultaneously to associate correlated sensor readings due to object motion and to filter out false positives from the sensors. A naive model of the PIR sensor is that it can detect all objects within a certain radius. However, this model is clearly not accurate for several reasons. First, each PIR sensor may be slightly different due to manufacturing or assembly process variations. Second, each node has four PIR sensors, one in each of the cardinal directions, and should be more likely to detect objects directly in front of one of the sensors than in one of the corners. Third, the analog output of the PIR sensor depends on the speed, size, distance, and direction of the object. Therefore, the node should be more likely to detect close objects, large objects, and objects moving quickly in a direction perpendicular to one of the sensors. Finally, because PIR sensors pick up any motion, they are very sensitive to nearby grass and wind. The actual response function of this sensor is much more complex than the naive model of detection, making it very difficult to predict how the MCMCDA algorithm would respond to an object as it takes a particular path through the deployment. To address this problem, we extended the empirical profiling techniques described in Chapter 4 to collect a thorough and complete empirical profile of our PIR sensor. The goal was

131 120 (a) Motion Sensor (b) Tracking Deployment Figure 7.1: A Tracking Deployment a) nodes with passive infrared (PIR) motion sensors depicted are deployed (b) in a 557 network in Berkeley, California.

132 121 to collect empirical data to represent the sensor response to an object moving near it in any direction and in any environment. We designed the following data collection process: We take four nodes and place them in a 10x10 foot square, each node at a slightly different orientation. We consider the first node to define a reference coordinate system, and to be oriented by definition at 0 degrees. The other three nodes are therefore located at 22 degrees, 44 degrees, and 66 degrees, respectively. This layout is represented by the squares in Figure 7.2(a). In the location and at the orientation of each of these nodes, we place three nodes at different elevations. One node is placed below grass level, one at grass level, and one above grass level. Thus, 12 nodes were deployed in total. One object passed through the sensing ranges of these nodes in a simple lawn mower pattern, moving up and down in parallel tracks with 5 foot spacing, as shown by the lines in Figure 7.2(a). At the end of each line, the position of the object was marked so that it could be correlated in time with the sensor responses. Thirteen 60 foot tracks were used to cover a 60x60 foot area. This experiment was repeated at three speeds of 3, 5, and 7 meters per second. After this was completed, the 12 nodes were re-deployed with a different permutation of node ID to node position. Each pass of the object in this experiment measured the response of a sensor to a single motion vector at every point in the sensor s two-dimensional coverage area. However, because each node has four PIR sensors, this pass actually measured four different motion vectors at each point. Furthermore, because we placed nodes at four different orientations, we were able to capture a total of 16 motion vectors at every point in space. These motion vectors were measured at three different elevations and, because the experiment was repeated with different permutations of nodes to node positions, the experiment also captured variation in node response due to hardware variations. Thus, by using multiple nodes simultaneously and by exploiting symmetry in the hardware, we were able to capture almost a complete profile of sensor responses in little more than an hour. This experiment did not profile different sizes of objects, although this could easily be measured as well. The actual measured response of the PIR sensors for a single elevation and set of motion vectors is shown in Figure 7.2(b). Differences and similarities between the empirical data and the

133 122 (a) PIR Profiling (b) PIR Response Figure 7.2: Profiling the PIR Sensor a) required a special layout of 12 nodes and a single object moving up and down in parallel lines. (b) This procedure measured 16 motion vectors at every point in the two-dimensional coverage area of the sensor.