Crowdsourcing location information to improve indoor localization

Transcription

1 Research Collection Master Thesis Crowdsourcing location information to improve indoor localization Author(s): Rogoleva, Luba Publication Date: 2010 Permanent Link: Rights / License: In Copyright - Non-Commercial Use Permitted This page was generated automatically upon download from the ETH Zurich Research Collection. For more information please consult the Terms of use. ETH Library

2 Crowdsourcing Location Information to Improve Indoor Localization Master Thesis Luba Rogoleva Supervisor: Philipp Bolliger Prof. Friedemann Mattern Distributed Systems Group Institute for Pervasive Computing Department of Computer Science 30th April 2010

3 Copyright 2010 Distributed Systems Group.

4 Abstract Recent research has shown that location fingerprinting has many advantages when it comes to indoor localization using existing infrastructures. However, the main problem of location fingerprinting is the collection of large sets of fingerprints, which is costly in terms of time and effort. The Redpin indoor localization system utilizes a crowdsourcing approach where the users themselves can add fingerprints to the system, thus training and using the system at the same time. This research has two major goals. The first is to analyze the characteristics and thus implications of taking a large number of measurements over a long period of time. The second goal is to investigate various mechanisms for creating applications and incentives which motivate users to contribute fingerprints continuously and consistently. iii

5 iv

6 Zusammenfassung Gemäss neusten Forschungsergebnissen ist es vorteilhaft, Location Fingerprinting zu verwenden, wenn versucht wird, unter Benutzung vorhandener Infrastruktur eine Positionsbestimmung innerhalb eines Gebäudes durchzuführen. Jedoch muss dazu, unter grossem zeitlichem und finanziellem Aufwand, eine grosse Menge an sogenannten Fingerprints gesammelt werden. Das Redpin Indoor Localization System verwendet einen Ansatz, bei dem Benutzer selbst Fingerprints zum System hinzufügen können. Dabei kann das System gleichzeitig verwendet und trainiert werden. Diese Arbeit hat zwei Hauptziele. Das eine ist, Charakteristiken und Konsequenzen einer Methode zu erforschen, die über längere Zeit eine grosse Anzahl an Messungen durchführt. Das zweite Ziel ist mögliche Anwendungen und Mechanismen zu analysieren, die dem Benutzer Anreiz geben, regelmässig und korrekt Fingerprints zur System Datenbank hinzuzufügen. v

7 vi

8 Acknowledgments I would like to thank Professor Friedemann Mattern for the opportunity to conduct my research in his group and enriching me with valuable tips of how to approach and successfully complete the research. I would like to express my sincere gratitude to my supervisor Philipp Bolliger for his support, guidance, and encouragement in carrying out this work. I am grateful to the people who have participated in our user studies for their kind help. In addition, I would also like to thank my lab colleagues for their support and fun times. Special thanks to Yaniv Hamo for keeping me believing in myself throughout the way. vii

9 viii

10 Contents 1 Introduction 1 2 Improving Redpin Localization Accuracy Related Work Redpin Indoor Localization Algorithms Redpin Algorithm Bayesian Inference Localization Algorithm Based on Support Vector Machine Data Set MultiM Data Collecting Data WiFi Signal Study Statistical Characteristics Received Signal Strength Distribution over Time AP Visibility APs Signals Correlation Distance from AP Separation of Fingerprints Effect of user presence on RSSI Type of Mobile Device Summary Conclusion Evaluation of Localization Algorithms Data Set Evaluation Tool Accuracy Results Redpin Improvements SVM on Redpin Server Interval Labeling on Redpin Client Additional Improvements User Motivation Study Related Work Incentives ix

11 x CONTENTS 3.3 Hunt The Fox Architecture User Study Meex Use Case Architecture Conclusion Discussion Conclusion Future Work Challenges A Accuracy of the Algorithms for Increasing Datasets 61 A.1 Instant Labeling A.2 Interval Labeling B Instant vs. Interval Labeling 63 C Hunt The Fox Survey 65 D Affidavit 69 E Statement regarding plagiarism when submitting written work at ETH Zurich 71 List of Figures 73 List of Tables 76 Bibliography 78

12 1 Introduction WiFi-based indoor localization is an active research area for several years [2, 5, 6, 16, 19, 28]. There are two main approaches for collecting labeled measurements of received signal strength (RSSI), so called fingerprints for WiFi indoor localization. The expert approach [2, 28] and the crowdsourcing 1 approach [5] through users collaboration. The expert approach consists of offline data gathering and training phase and real-time location determination phase. During the offline phase, signal strength fingerprints from known access points are systematically collected into a radio map. Radio maps are used in the real-time phase to estimate location based on measurements received from the users. The main problem with the expert approach is resources in terms of time, effort, and cost. In addition, it requires continuous rebuilds of the radio map due to environmental changes. Moreover, expert-based systems have to provide radio maps for all locations in advance, which is not practically feasible. Crowdsourcing-based approaches, recently gaining popularity within the research community [5, 6, 22], perform both training and querying in real-time, requiring no offline phase. Signal strength measurements labeled by users are collected in a database of fingerprints and used to estimate the location of unlabeled measurements coming from other users. The estimation is done by calculating various similarity metrics between a given unlabeled measurement and the collection of fingerprints. Crowdsourcing approaches do not require offline training. However, since there is no systematic way of collecting fingerprints, it introduces new challenges such as figuring out how large a collection of fingerprints is needed for accurate location estimations, or the optimal freshness of the fingerprints collection. Another important challenge which is often overlooked is how to initially get fingerprints from the users and motivate them to continuously contribute their location. We further discuss these challenges and others later on. Location fingerprints can be collected by two different techniques: instant labeling and 1 Crowdsourcing is a neologistic compound of crowd and outsourcing for the act of taking tasks traditionally performed by an employee or contractor, and outsourcing them to a group of people or community, through an open call to a large group of people (a crowd) asking for contributions. (Wikipedia) 1

13 2 interval labeling[6]. With instant labeling, end-users label their location and instantly a WiFi measurement is taken by the mobile device. With interval labeling, pioneered by the PILS [6] system, measurements are continuously taken for the corresponding label as long as the device is stationary. Initial results of PILS suggest that the accuracy of localization algorithms can be improved by interval labeling. This work is mainly based on the Redpin project 2. In the first part of the work we analyze the characteristics of WiFi signals which were collected over extended period of time to study their properties. We then try to improve the accuracy of Redpin using interval labeling on the iphone. When examining as well as designing localization systems one has to consider the people who use it, particularly with a crowdsourcing-based approach. An essential part of crowdsourcing-based systems is to study and understand how to motivate users. In the second part of this work we experiment with ways to motivate users to contribute their location. Two applications with different characteristics were developed as part of this research: Hunt The Fox and Meex. We study the experience of users through Hunt The Fox. 2

14 2 Improving Redpin Localization Accuracy 2.1 Related Work Information about position indoors plays an essential role in context-aware systems. There are several major location sensing technologies, e.g. infrared, ultrasound, and radio frequency (RF) that are used for indoor localization systems. An example of a system using infrared technology is Active Badge [27] developed by Olivetti Research Ltd. Active Badge is a system for determining users location in an office environment. All people in the office wear a badge device which systematically transmits a unique infrared signal. The signal is detected by a network of sensors spread throughout the office and transmitted to a central location server. The server can then determine the location of a badge by proximity to the nearest receiver sensor. Cricket [24], an indoor localization system developed at MIT, is an example of a system based on ultrasound technology. Cricket has no central database or controller, but uses a network of mobile devices as passive listeners. The devices measure ultrasound and RF signals coming from pre-mounted active beacons, and determine their location via triangulation from at least three active beacons. The advantage of Cricket is that it is decentralized, thus allowing for a high level of user privacy. Both infrared and ultrasound systems require specific hardware equipment to be installed and present, making their use quite limited. In addition, infrared light signal requires a clear line-of-sight between the transmitter and receiver, which is typically not the case in dynamic environments. Therefore, many indoor localization systems use technologies based on RF and utilizing existing hardware, such as Bluetooth signals [15] or GSM signals [11]. In recent years, the most popular technology used by localization systems [2, 5, 16, 28] is wireless LAN (WiFi, IEEE ) because these networks are ubiquitous and almost every mobile device has a built-in wireless equipment these days and can be used for location detection without carrying around or installing any special hardware device. Another benefit of using 3

15 RELATED WORK radio signals is that they traverse through different materials - an advantage in dynamic environments. RADAR [2], developed by P. Bahl and V. N. Padmanabhan, is an example of one of the first wireless RF based systems for locating and tracking users indoor. Localization systems based on RF technology fall into two categories [14]: systems which provide physical position, such as Cartesian coordinates, and systems that provide symbolic information, such as unique room labels. The former use geometry properties to estimate users location (e.g. Place Lab [19]), and the latter identify users location based on proximity to already known locations (e.g. Redpin [5] and Nibble [7]). While systems providing physical position have good accuracy, they require advance training and so their setup and maintenance costs are high. In addition, room-level positioning, rather than the exact location on Cartesian coordinates, would be enough for most of the indoor localization systems. This work uses symbolic tags as locations identifiers, and focuses on room-level positioning. As mentioned in the introduction, Location Fingerprinting (LF) is a promising method which does not require special hardware deployment and draws increasing attention by the research community. The underlying idea behind LF is to determine user location by comparing RSSI measured in their device with a set of fingerprints collected earlier. Kjaergaard compares different LF systems and proposes taxonomy for their better understanding [18]. However location determination using LF is not a trivial problem due to the unstable environment a wireless signal has to travel through. The radio path between a transmitter and a receiver is affected by many factors [17], among which are the presence of humans, device and user orientation, interferences, time of day, presence of obstacles and walls, their material and layout, and weather. The combined effect is multi-reflection, multi-diffraction, and multi-scattering [4]. In the course of this work we extensively study some of these factors and characterize their influence on WiFi signals. LF-based techniques belong to one of two groups: techniques which use deterministic algorithms to estimate users location and techniques which use probabilistic algorithms. Kjaergaard [18] classifies them as deterministic estimation methods and probabilistic estimation methods respectively. Table 2.1 shows a few representative algorithms which are used by these estimation methods [18]. We compare the highlighted algorithms later on. Deterministic estimation methods Neural Network Nearest Neighbor Trilateration Offset Mapping Support Vector Machine Hillclimbing Search Probabilistic estimation methods Discrete Space Estimator Center of Mass Particle Filter Graphical Models Bayesian Inference Markov Chain Hidden Markov Model Table 2.1: Estimation methods [18]. We compare the highlighted algorithms later on. RADAR [1, 2] is an example of a system utilizing deterministic techniques, in this case k- Nearest Neighbors (knn) algorithm. Probabilistic techniques, giving good performance and becoming increasingly popular [8], are adopted by systems such as Horus [28] and Nibble [7] which use Bayesian networks, and PILS [6] which uses Bayesian inference method assuming normal distribution of the signal strength.

16 CHAPTER 2. IMPROVING REDPIN LOCALIZATION ACCURACY 5 Redpin [5] is using Nearest Neighbor, a deterministic estimation method. In order to improve Redpin, we compare the performance of the original algorithm with two modified versions: one where we replace NN with another deterministic method, Support Vector Machine, and one where we utilize a probabilistic method based on Bayesian Inference. Closely related works include the WASP algorithm [22] which improves Redpin for congested WiFi environments. WASP is based on a limited set of around 1000 unique fingerprints, and interval labeling and long-term WiFi measurements were not used. Moreover, performance in terms of running time was not considered during the development of WASP. In contrast, our research is based on a set of about fingerprints, and we include a performance analysis of the algorithms. General comparison of knn and probabilistic methods in terms of accuracy, performance and storage space requirements can be found in [21]. Another work compares NN with an algorithm based on support vector machine, and one based on Gaussian Process in indoor environment [11]. However they make use of GSM fingerprints as opposed to WiFi signals. Zhen and Jia [29] also propose an algorithm based on support vector machine with high accuracy. An extensive study of the statistical properties of WiFi signals has been carried out by K. Kaemarungsi [17]. The study collects measurements using a quasi-static laptop (at one of predefined locations) while we use an iphone device to collect measurements from arbitrary locations chosen by users. We refer to Kaemarungsi s research later in our work. The rest this first chapter of the work is organized as follows. Section 2.2 gives a brief overview of the Redpin system. In Section 2.3 we list the various indoor localization algorithms that were considered for evaluation. Section 2.4 describes how the long-term measurement experiment was conducted, and based on the collected measurements we present an extensive WiFi signal study in Section 2.5. We evaluate the algorithms in Section 2.6 and present improvements of Redpin localization in Section 2.7 based on the results. 2.2 Redpin Redpin is an LF-based indoor localization system designed and built to run on mobile phone clients in a client-server architecture. Currently the client comes in two versions - J2ME Client and iphone Client. Server Mobile Phone Locator JSON Sniffer UI Figure 2.1: Redpin system architecture.

17 INDOOR LOCALIZATION ALGORITHMS Redpin s architecture (Figure 2.1 [5]) consists of two basic parts. The Sniffer component running on the mobile phone measures the RSSI of all WiFi access points in the area of the device to create a fingerprint. The Locator component on the server is using this fingerprint as an input to the location estimation process. The Locator uses the Redpin algorithm (described in Section 2.3.1) which follows the basic LF approach - it estimates a location by performing knn on the measured RSSI values received from the mobile device, and a collection of fingerprints stored on the server. Communication between the server and client is done through asynchronous JSON messaging using a polling mechanism. The current version of the server is implemented in Java and uses SQLite for the database. We would like to improve the accuracy and performance of the Locator component at server side, as well as implement interval labeling on the client to provide the Locator with data of higher quality. 2.3 Indoor Localization Algorithms In the course of this research we have evaluated three algorithms Redpin Algorithm The Redpin algorithm is a deterministic algorithm based on knn where k=1. To predict the location of a target WiFi measurement it calculates an AP-similarity level [5] between the target measurement and every measurement stored in the database. The location of the most similar measurement from the database is used as the output for the algorithm. AP-similarity level is calculated as follows. Every Access Point (AP) seen in both the target and a database measurement contributes a bonus factor to the similarity level. Every AP not shared by both measurements receives a penalty value which is reduced from the similarity level. In addition, for every matching pair of APs a signal contribution is calculated based on similarity between their RSSI values and added to the total similarity level Bayesian Inference The basic idea behind probabilistic location estimation methods is to store information about RSSI distributions and use probabilistic inference algorithms to calculate the location for a given RSSI measurement. Probabilistic methods, therefore, do not require keeping a database of all collected fingerprints, which can be costly in terms of resource allocation, but rather keep only a few values that characterize the RSSI distribution, such as mean and variance. When a new fingerprint arrives, the probabilistic based system only updates the existing mean and variance without storing the fingerprint string. In addition, such methods can save calculation time considerably, since estimating a location from the distribution function is notably faster than comparing the RSSI values against all fingerprints in the database (as does the current version of Redpin). We use Bayesian inference to compute the distribution conditioning on the RSSI. To implement our probabilistic algorithm we use the method adopted by PILS [6] which assumes normal distribution of RSSI (received signal strength indicator) values. Formally, we model each WiFi measurement as a vector of n pairs:

18 CHAPTER 2. IMPROVING REDPIN LOCALIZATION ACCURACY 7 M t = (BSSID 1t, RSSI 1t ),..., (BSSID nt, RSSI nt ), Where every tuple (BSSID it, RSSI it ) indicates signal strength from an access point with unique identifier BSSID i at time t. Every fingerprint is represented by a tuple F t = (L k, M t ), where L k is an unique symbolic id of a location corresponding to F t and L k L = {L 1,..., L J } - a set of all known locations. To estimate a location symbolic id L of a target measurement M t we use: L = argmax Lk P (L k M t ) = argmax Lk P (M t L k ) P (L k ) P (M t ) = argmax Lk P (M t L k ) P (L k ) We assume uniform distribution of all possible locations, hence P (L k ) = 1 L, which is constant and can be omitted from the calculation. The conditional probability that WiFi measurement is taken at location L can be written as follows, assuming signal strengths from different access points are not correlated (this is demonstrated in our WiFi signal study in the Section 2.5.4): P (M t L k ) = P ((BSSID 1, RSSI 1 ),..., (BSSID I, RSSI I ) L k ) = I P ((BSSID i, RSSI i ) L k ) i=1 That is, in order to calculate the probability that measurement M t has been taken in location L k we multiply all the probabilities that access point BSSID i has signal strength RSSI i at location L k. We calculate that probability assuming normal distribution of signal strength values at any access point [6]: P ((BSSID i, RSSI i ) L k ) = P Lk (BSSID i ) N ( RSSI i ; µ Lk (BSSID i ), σ 2 L k (BSSID i ) ) Localization Algorithm Based on Support Vector Machine Support Vector Machine (SVM) is a statistical learning technique widely used for data classification [9]. Data classification is a machine learning procedure that groups individual items of a dataset based on one or more element characteristics, called attributes. SVM receives a training set of labeled data as input and builds a model which is then used to predict whether a given element belongs to one classified category or another based on its attributes. Implementation of our SVM approach is based on LIBSVM 1 - a software library for support vector classification. Formally, every fingerprint F t = (L k, M t ) stored in the database is represented as a data item used by SVM to build the prediction model. Every symbolic id of a location L i specifies a different classification group of L groups in total. The number of categorical attributes is the number of all unique access points in the database, i.e. BSSID = {BSSID 1,..., BSSID N }. 1 cjlin/libsvm/

19 DATA SET Thus, the fingerprint F t = (L k, (BSSID 1t, RSSI 1t ),..., (BSSID nt, RSSI nt ) ) belongs to classification group L k has (BSSID 1,..., BSSID n ) as attributes and (RSSI 1,..., RSSI n ) as values corresponding to these attributes. SVM is then trained on the dataset of fingerprints and creates a model which given an input set of attributes and values corresponding to them (i.e. given a measurement), predicts the classification group (i.e. the symbolic id of the location where the measurement was taken). The advantage of SVM is that even though it is a deterministic approach, it takes little time to perform the predicition calculation once the classification model is built. 2.4 Data Set To better understand the characteristics of WiFi signals, we conducted a long-term WiFi measurement. For this purpose we developed an iphone application which scans WiFi signals, and distributed the application among several people who were running the application for a total of 6 weeks. We then proceeded to study the collected data and compared the three localization algorithms described in the previous section for different setups. Experiments are based on room-level location detection MultiM To collect WiFi readings we implemented a simple client-server application called MultiM (Figure 2.2), where the client is running on iphone. MultiM s client has one input field where end-users can label their current location. Upon clicking the Go button MultiM starts scanning and measuring WiFi signals strengths from all surrounding access points every 30 seconds. These measurements are sent together with the location tag to the Redpin server and stored as a fingerprint in the database Data Collecting To get as many fingerprints as possible we asked users to help and participate in collecting the data. We also believe that the approach of collecting measurements from users as opposed to experts is more realistic. Users collect data from locations they visit, for the time they spend in those locations, while they place their mobile device at arbitrary spots in these locations. Algorithms using fingerprints contributed by end-users would most likely have to deal with data collected in a similar fashion, as opposed to data collected in a systematic way such as an identical number of measurements taken in every room. Figure 2.2: iphone. MultiM user interface on

20 CHAPTER 2. IMPROVING REDPIN LOCALIZATION ACCURACY 9 Fourteen users were collecting measurements. Each user installed the MultiM application on their iphone. Nine users had iphone 3GS, five had iphone 3G and one user had the second generation iphone 2G. Users were divided into two groups: the first group of 10 users was running the application at the ETH campus in two adjacent building IFW and RZ; the other group of 4 users consisted of people working at the same private office and agreed to help us taking measurements from their work place. The first group we call the campus group, the second - the office group. In addition, two users were taking measurements from their private residence. We divided the six weeks of collection of data into two periods of three weeks each. During the first half all users were asked to keep their mobile device at the same spot in their current location while running MultiM. During the second half users were asked to place their mobile device in different spots in their current location thus allowing us to evaluate differences between fixed and moving receivers Data In total unique measurements were collected by the users from 23 different locations: measurements from fixed receivers in 19 locations and measurements from moving receivers in 16 locations. Room Number of measurements R R R R Table 2.2: Distribution of measurements from fixed receivers collected by the office group. Figure 2.3: Floor plan of the office. The red dots are the location of mobile devices while taking measurements. The distribution of measurements from fixed receivers in the office group is shown in Table 2.2. The office group has been collecting measurements from their working desks from two

21 DATA SET non-adjacent rooms R1 and R2. Three users were sitting in the shared open-space R1 and one user in room R2, as shown in Figure 2.3. Even though the 3 users sitting in the same room were not divided by walls, we give their positions different location tags (R1.1 - R1.3) due to the large size of the room (about 100 square meters). We found it interesting to study the behavior of WiFi signals in such an environment. Information about the locations of access points in the office was not made available to us. Also, we did not receive any measurement entries for moving receivers from the office group since they were all out of the office during Christmas and New Year s holidays. Room Number of measurements IFW A IFW A IFW B IFW D IFW D IFW D IFW D IFW D RZ F RZ H RZ H2 907 Table 2.3: Distribution of measurements from fixed receivers collected by the campus group. Room Number of measurements IFW A IFW A32 52 IFW B IFW D IFW D IFW D IFW D IFW D IFW D IFW D42 63 IFW D RZ F RZ H2 548 Table 2.4: Distribution of measurements from moving receivers collected by the campus group. The distribution of fixed receiver measurements from the campus group is shown in Table 2.3. The data was collected from two adjacent buildings IFW and RZ. In IFW users took

22 CHAPTER 2. IMPROVING REDPIN LOCALIZATION ACCURACY 11 measurements from 3 different floors A, B, and D and 8 different rooms: two rooms A44 and A36 on floor A (Figure 2.7); one room B47.1 on floor B (Figure 2.7) and 5 rooms D35.1, D43.1, D44, D47.1, and D47.2 on floor D (Figure 2.4). In RZ building users took measurements from 3 different locations: rooms H1.1 and H2 on floor H (Figure 2.5) and room F13 on floor F (Figure 2.6). The distribution of moving receiver measurements from the campus group is shown in Table 2.4. The data was collected from two adjacent buildings IFW and RZ. In IFW users took measurements from 3 different floors A, B, and D and 11 different rooms: two rooms A44 and A32 on floor A (Figure 2.7); one room B47.1 on floor B (Figure 2.8) and 8 rooms D35.1, D43.1, D44, D47.1, D47.2, D41.2, D42, and D41.1 on floor D (Figure 2.4). In RZ building users took measurements from 2 different locations: room H2 on floor H (Figure 2.5) and room F13 on floor F (Figure 2.6). Figure 2.4: IFW D floor plan. The dots are rooms with known access points routers. Rooms where measurements were taken from both fixed and moving receivers are marked green. Rooms where measurements were taken from moving receivers are marked red. Figure 2.5: RZ H floor plan. The dots are rooms with known access points routers. Rooms where measurements were taken from both fixed and moving receivers are marked green. Rooms where measurements were taken from fixed receivers are marked yellow.

23 DATA SET Figure 2.6: RZ F floor plan. The dots are rooms with known access points routers. Rooms where measurements were taken from both fixed and moving receivers are marked green. It is important to point out that every wireless physical AP in IFW and RZ buildings broadcasts with multiple Basic Service Set Identifiers (BSSID - the MAC address of the wireless access point), thus functioning as several virtual APs. Figure 2.7: Relevant part of the IFW A floor plan. The dots are rooms with known access points routers. Rooms where measurements were taken from both fixed and moving receivers are marked green. Rooms where measurements were taken from moving receivers are marked red. Rooms where measurements were taken from fixed receivers are marked yellow. Figure 2.8: Relevant part of the IFW B floor plan. The dots are rooms with known access points routers. Rooms where measurements were taken from both fixed and moving receivers are marked green. The distribution of measurements from fixed receivers and from moving receivers collected by the users in their private residences are shown in Tables 2.5 and 2.6 respectively. Two users were collecting data in their private residences: one user from two different locations Home 1.1 and Home 1.2 ; and another user from three different locations Home 2.1, Home 2.2, and Home 2.3. The first user did not collect data from a moving receiver. The floor plans of the private residences were not made available to us. The only provided information is that room Home 2.1 is adjacent to room Home 2.2 which is adjacent to the

24 CHAPTER 2. IMPROVING REDPIN LOCALIZATION ACCURACY 13 Room Number of measurements Home Home Home Home Room Number of measurements Home Home Home Table 2.5: Measurement distribution from fixed receivers collected by users in their private residences. Table 2.6: Measurement distribution from moving receivers collected by users in their private residences. room Home 2.3. Rooms Home 2.1 and Home 2.3 are not adjacent. Positions of access points are unknown. 2.5 WiFi Signal Study We analyzed the characteristics and behavior of the WiFi measurements with the goal of understanding the properties of WiFi signals collected indoors over a long term, and their variance over time. This forms the basis required for improving indoor localization algorithms relying upon signal strength (RSSI). As mentioned earlier, physical APs in IFW and RZ buildings broadcast as several virtual APs. Before starting our analysis we wanted to check whether virtual APs broadcasted by the same physical AP can be considered as one AP in our study. This simplification would be possible if all virtual APs transmit the same signal strength. Figure 2.9: RSSI from virtual APs broadcasted by a single physical AP located in room IFW A36 as measured in room IFW A44 during a 15 minutes period.

25 WIFI SIGNAL STUDY Figure 2.9 shows RSSI from the virtual APs of a single physical AP located in room IFW A36 as measured by a mobile phone in room IFW A44 during a 15 minutes period. It can be seen that RSSI from virtual APs of the same physical AP are not only different most of the time, but can vary quite significantly. For example, at time 15:39:00 the difference between RSSI received from virtual AP with BSSID 0:3:52:1c:13:60 and RSSI received from virtual AP with BSSID 0:3:52:1c:13:63 is 7dBm. It can also be seen from Figure 2.9 that even though all 4 signals are transmitted from one physical AP, not all of them are always captured by the mobile device. Following these observations we decided to consider each virtual AP as a physical one for the course of the WiFi signal study Statistical Characteristics In most indoor environments the existence of a direct line-of-sight (LoS) path between the transmitter and the receiver is highly unlikely. Therefore, the wireless signal mainly propagates along different paths of varying lengths due to scattering and reflection from the obstacles inside a building, by diffraction over or around them [4]. At the receiver, the resulting signal of these multipath waves fluctuates in time and space. This phenomenon is known as multipath fading effect or small-scale fading at the receiver. These fluctuations in the signal amplitude explain why RSSI at one location can be very different from RSSI in a close by location. Another known physical phenomenon, namely absorption of the signal by different materials inside a building, causes long term variations in the mean signal level and is called large-scale fading. We calculated the mean and standard deviation at different locations for all APs signals and for both fixed and moving receivers. A few observations can be made by examining the results which are partially given in Tables 2.7 and 2.8. First of all, the measured RSSI is lower the farther the receiver is located from the AP, due to the path loss propagation property of the wireless signal. For example, AP with BSSID 0:3:52:4d:e7:90 is located in room IFW D42, RSSI from this AP measured in room IFW D35.1 has a mean of dBm throughout the whole period the fixed receiver was recording (Table 2.7), while RSSI measured in room IFW D43.1 has a mean of dBm for the fixed receiver (Table 2.8). Room IFW D43.1 is much closer to room IFW D42 where the AP is located, than room IFW D35.1 (can be seen in Figure 2.4). The same can be observed for AP 0:3:52:1c:31:60 which is located in room IFW D46.2, and the other APs with known locations which we checked. However, we have also observed that the signal propagation is not only a function of distance between the transmitter and the receiver - room IFW D35.1 is closer to the AP in room IFW D42 than to the AP in room IFW D46.2 (Figure 2.4) - but the mean of RSSI measured in room IFW D35.1 for both fixed and moving receiver is lower for the former AP than for the latter. This implies that if a receiver is closer to AP A than it is to AP B, the RSSI from AP A is not necessarily higher than the RSSI from AP B at the same position (Tables 2.7 and 2.8) because of the different transmitting power of the specific access point and fading effects. Another observation we made is that the standard deviation is higher for the moving receiver than for the fixed receiver at the same location and from the same AP. We found that the maximal standard deviation for a moving receiver over all locations and APs is 9.52dBm

26 CHAPTER 2. IMPROVING REDPIN LOCALIZATION ACCURACY 15 0:3:52:4d:e7:90 (IFW D42) 0:3:52:1c:31:60 (IFW D46.2) IFW D35.1 Fixed receiver Moving receiver Fixed receiver Moving receiver Mean Std dev Table 2.7: Mean and standard deviation as measured during the whole period by fixed and moving mobile devices in room IFW D35.1 from AP with BSSID 0:3:52:4d:e7:90 located in room IFW D42 and AP with BSSID 0:3:52:1c:31:60 located in room IFW D :3:52:4d:e7:90 (IFW D42) 0:3:52:1c:31:60 (IFW D46.2) IFW D43.1 Fixed receiver Moving receiver Fixed receiver Moving receiver Mean Std dev Table 2.8: Mean and standard deviation as measured during the whole period by fixed and moving mobile devices in room IFW D43.1 from AP with BSSID 0:3:52:4d:e7:90 located in room IFW D42 and AP with BSSID 0:3:52:1c:31:60 located in room IFW D46.2. and for a fixed receiver is 7.758dBm. To increase the statistical validity of our results we considered only locations where a signal from a particular AP has been seen at least 100 times. The average value of standard deviations for a fixed receiver is 2.444dBm and for a moving receiver is 2.684dBm. Intuitively, when a receiver is moved around the room, the received signals vary more than when it is placed statically due to the multipath effect which can cause several fades in a short duration. For example, in Tables 2.7 and 2.8 for all APs and locations, the standard deviation for moving receivers is higher than for fixed ones. However, mean values are not varying by much between fixed and moving receivers. As mentioned above, the mean of RSSI is influenced more by the large-scale fading effect, caused by absorption of signal by materials such as walls and floors, which can be assumed to not change much over short periods of time. However, in the example shown in Table 2.8, the mean value of RSSI measured in room IFW D43.1 from AP 0:3:52:4d:e7:90 for the moving receiver is almost 10dBm lower than for the fixed receiver. This can be explained by the relatively short distance between the AP located in room IFW D42 and the receiver in room IFW D43.1, since, as explained in the next paragraph, the closer the transmitter is to the receiver, the higher the fluctuations. Another known property of WiFi signals, is that the lower the signal strength, or the farther an AP is from the receiver, the smaller the standard deviation of RSSI and so the fluctuations [17]. And vice versa, the higher the signal strength, or the closer an AP is from the receiver, the larger the standard deviation and the fluctuations. Indeed, we observed this property in most of the cases - examples are given in Tables 2.9 and However it is not always the case for moving receivers (Table 2.11). The reason could be that in one location the receiver has been moved around more frequently and placed in more spots than in the other location (the rooms are almost of the same size). Again, these observations show us how unpredictable the WiFi signal could be in a dynamic environment.

27 WIFI SIGNAL STUDY Mean Std dev IFW D IFW D Mean Std dev RZ H RZ H Table 2.9: Mean and standard deviation as measured during the whole period from AP with BSSID 0:3:52:4d:e7:91 located in room IFW D41.2 by fixed mobile devices in rooms IFW D35.1 and IFW D43.1. Table 2.10: Mean and standard deviation as measured during the whole period from AP with BSSID 0:3:52:1c:12:80 located in room RZ H9 by fixed mobile devices in rooms RZ H1.1 and RZ H2. Mean Std dev IFW D RZ H Table 2.11: Mean and standard deviation as measured during the whole period from AP with BSSID 0:3:52:5c:27:0 located in room RZ G2 by moving mobile devices in rooms IFW D41.2 and RZ H Received Signal Strength Distribution over Time Most of the previous works assume that RSSI is lognormally distributed and that the fingerprint distributions stay unchanged over time. However, in very dynamic environments this assumption might not hold due to the fading effects mentioned above and continuous changes such as movement of people (we discuss the effect of users presence over RSSI in Section 2.5.7). Figure 2.10: Histogram of RSSI as recorded by a fixed receiver at location IFW A44 for a period of 21 days from AP 0:3:52:1b:f6:2 and corresponds to a normal distribution with a mean of and standard deviation Figure 2.11: Histogram of RSSI as recorded by a moving receiver at location IFW A44 for a period of 17 days from AP 0:3:52:1b:f6:2 and approximates a normal distribution with a mean of and standard deviation

28 CHAPTER 2. IMPROVING REDPIN LOCALIZATION ACCURACY 17 In many past studies of RSSI distribution, samples were taken in a systematic way. For instant, a study based on five second sampling over an extended duration of five hours, 20 hours and one month might conclude that RSSI is lognormally distributed [17]. However, this approach is not realistic for systems based on crowdsourcing, since users are unlikely to contribute fingerprints in a systematic way. Our results show that the distribution of RSSI may vary significantly between different times of day and duration, thus a localization algorithm should not depend on a specific RSSI distribution. Figure 2.10 shows the histogram of RSSI as measured by a fixed receiver at location IFW A44 for a period of 21 days from AP 0:3:52:1b:f6:2 (2945 unique readings in total). Figure 2.11 shows the histogram measured by a moving receiver at the same location from the same AP for a period of 17 days (2200 unique readings in total). The differences between the histograms are clearly visible, the fixed receiver having an almost perfect normal distribution. One interesting observation is that for both histograms the maximum is at the same value - 84dBm. Figure 2.12: Histogram of RSSI as recorded by a fixed receiver at location IFW A44 for a period of 21 days from AP 0:3:52:4e:07:d1. It corresponds to a normal distribution with a mean of and standard deviation Figure 2.13: Histogram of RSSI as recorded by a moving receiver at location IFW A44 for a period of 17 days from AP 0:3:52:4e:07:d1 and corresponds to a normal distribution with a mean of and standard deviation We now consider the differences in signal distributions at the same location IFW A44, but from AP 0:3:52:4e:07:d1 located at room IFW A34 which is close to the receiver - that is, the transmitter and the receiver are in adjacent rooms (Figure 2.7). Figure 2.12 shows RSSI histogram as measured by a fixed receiver (4246 unique readings in total) and Figure by a moving receiver (4012 unique readings in total). In both cases the RSSI distributions are similar. As in the previous example, both histograms for the fixed receiver and for the moving receiver have a maximum at the same value -56dBm. It is interesting to note that while there is no time overlap between the measurements by the fixed and moving receivers both show quite a few RSSI readings at -74dBm. We checked the days and times when the value -74dBm was measured and did not find any particular pattern - it was measured almost every day at different times of the day, earliest at 10:18 in the morning and latest at 21:49 in

29 WIFI SIGNAL STUDY the evening. We therefore attribute this behavior to the transmitting power of the AP. All histograms in Figure are smooth and can be approximated by lognormal distribution, and this holds for the majority of cases when the measurement period is at least one day. However, we also found a few examples of histograms which can not be approximated by the lognormal distribution. Examples are given in Figures Figure 2.14: Histogram of RSSI as recorded by a fixed receiver at location Home 1.2 for a period of 20 days from AP 0:1f:f3:64:b3. Figure 2.15: Histogram of RSSI as recorded by a fixed receiver at location IFW D35.1 for a period of 21 days from AP 0:3:52:1c:31:60. Figure 2.16: Histogram of RSSI as recorded by a fixed receiver at location IFW D35.1 for a period of 21 days from AP 0:3:52:1c:31:63. Figure 2.17: Histogram of RSSI as recorded by a moving receiver at location IFW D41.2 for a period of 14 days from AP 0:3:52:4d:e7:93. These results can be explained by non-systematic data collection. Users who collected measurements were not following any specific requirements other than keeping the mobile device static or to move it around different spots at the same location. They therefore were not, for example, limited to a specific time of day or to a certain length of measurement period. They took measurements in unpredictable ways, and the histograms illustrate just how arbitrary the data could be for a crowdsourcing-based system. Until now we looked on RSSI distributions over the whole measurement period as well as for both fixed and moving receivers. To complete the analysis, we compare and check other

30 CHAPTER 2. IMPROVING REDPIN LOCALIZATION ACCURACY 19 lengths of time periods to investigate how WiFi signals change within a day and from day to day in the following. Figure 2.18: Histogram of RSSI as recorded by a fixed receiver at location IFW A44 for a period of 2 hours between 20:28 and 22:27 on the first day of measurement from AP 0:3:52:1b:f8:30. Figure 2.19: Histogram of RSSI as recorded by a fixed receiver at location IFW A44 for a period of 3 hours between 13:40 and 16:40 on the sixth day of measurement from AP 0:3:52:1b:f8:30. Figure 2.20: Histogram of RSSI as recorded by a fixed receiver at location IFW A44 for a period o 2 hours between 17:17 and 19:14 on the last day of measurement from AP 0:3:52:1b:f8:30. Figure 2.21: Histogram of RSSI as recorded by a fixed receiver at location IFW A44 for a period of 3 hours between 13:24 and 16:30 on the seventh day of measurement from AP 0:3:52:1b:f8:30. Figures show the variation between RSSI distributions as measured by a fixed receiver located in room IFW A44 from AP 0:3:52:1b:f8:30 for periods of two to three hours, and during different days and different times of day. It can be seen how RSSI distributions dramatically change, invalidating the common assumption that data distribution remains unchanged over time. This assumption does not hold even for RSSI distributions measured over the same time of day, at the same location and AP, for two consecutive days as seen in Figures 2.19 and A localization algorithm which works under the assumption of lognormal distribution as in Figure 2.21, would demonstrate poor accuracy for the period shown in

31 WIFI SIGNAL STUDY Figure In the following we compare RSSI distributions from the same AP over the same time period of one working day (from 9:34 in the morning to 16:39 in the afternoon) as measured by fixed receivers at two different locations in the open spaced office (Figure 2.3). Figure 2.22 shows the RSSI histogram at location R1.1 and Figure 2.23 shows the RSSI histogram at location R1.3. We believe that the differences in distributions originate from the location characteristics of R1.1 and R1.3 within the office. R1.1 is a corner with only a few employees passing by during the day, while R1.3 is located just next to the corridor which is constantly crowded. The RSSI distribution at R1.1 is almost a perfect normal distribution and at R1.3 is more chaotic as it is affected by the constantly changing environment. Once more, these results demonstrate the unpredictability of WiFi signals behavior due to changes in the environment. Figure 2.22: Histogram of RSSI as recorded by a fixed receiver at location R1.1 for a period of one working day on the fifth day of measurement from AP 0:23:df:f8:ba:6b. Figure 2.23: Histogram of RSSI as recorded by a fixed receiver at location R1.3 for a period of one working day on the fifth day of measurement from AP 0:23:df:f8:ba:6b AP Visibility Due to path loss of the RF wave as it propagates through space, there is a limited number of APs seen at each location. In addition, fluctuations and fading effects cause certain signals to not be captured temporarily by the mobile device, but captured at different time. Therefore in a dynamic environment the number of APs seen by a mobile device at a certain location is not constant over time. Figure 2.24 shows an example of how the number of APs measured by a moving receiver changes over time during two and a half hours in the afternoon, at one of the campus locations. There are many APs throughout the campus, and each broadcasts several virtual APs (typically 4-6), hence the large number of APs in the histogram. Figure 2.25 shows how the number of APs changes over time for fixed receivers during four hours at night as measured at one of the campus locations. Even at night time when, we assume, there are no people around, the number of APs captured by the mobile device is constantly changing. We proceeded by analyzing how the number of physical APs (as opposed to virtual ones) changes over time for the same periods. The results are shown in Figures 2.26 and When we consider only physical APs the picture is completely different, and better reflects

32 CHAPTER 2. IMPROVING REDPIN LOCALIZATION ACCURACY 21 Figure 2.24: Number of virtual APs over time, as measured by a moving receiver at IFW D43.1 during 2.5 hours in the afternoon. Figure 2.25: Number of virtual APs over time, as measured by a fixed receiver at IFW D41.2 during 4 hours at night. our expectations. At night (Figure 2.27) the number of APs seen by the mobile device stays almost constant, while during day time it fluctuates. Therefore we conclude that the physical APs continuously change the number of virtual APs they broadcast themselves as. However, algorithms used by LF-based systems have no apriori information about the radio map of APs, and cannot tell which virtual AP is being broadcasted from which physical AP. These algorithms consider any unique BSSID as different AP. Our study demonstrates that we cannot assume that mobile devices see a constant, or relatively static number of APs, even during quiet times without much environmental changes like over night. Figure 2.26: Number of physical APs as measured by a moving receiver at IFW D43.1 during 2.5 hours in the afternoon. Figure 2.27: Number of physical APs as measured by a fixed receiver at IFW D41.2 during 4 hours at night. As mentioned above, due to long-term fluctuations and short-term variations of WiFi signals in indoor environments some APs have higher visibility at a certain location than other APs.

33 WIFI SIGNAL STUDY Several localization algorithms give weights to APs at certain location depending on how often this AP has been seen in this location, and perform filtering of APs based on that weight [22]. In the following study our goal is to investigate how the visibility of APs changes over time. We consider ten classes of AP visibility: from being seen in 0-10% of the measurements to being seen in % of the measurements. We calculated, for each class, what portion of the APs has that visibility during a certain period of time. The results for room IFW A44 and three time periods of different lengths are shown in Figures Figure 2.28: Distribution of AP visibility over two and a half hours period. For example, Figure 2.28 shows the distribution of AP visibility over a period of 2.5 hours (from 10:30 to 13:00). APs that are visible in 90%-100% of the measurements constitute the majority. APs that are seldom visible, in 10%-20% of measurements, constitute less than 5% of all visible APs. Figure 2.29: Distribution of AP visibility over 5 days period. Figure 2.30: Distribution of AP visibility over 16 days period. As the period of measurement increases, a different picture emerges. The number of uncommonly seen APs grows. For a measurement period of 5 days APs that are visible in 10%-20%

34 CHAPTER 2. IMPROVING REDPIN LOCALIZATION ACCURACY 23 of measurements constitute more than 20% (Figure 2.29), and during 16 days - more than 50% (Figure 2.30). This shows how much noise is captured by a mobile device during a long period of time and confirms that filtering techniques of APs with lower visibility could be a good approach in improving accuracy. However, filtering should be done carefully. We checked the specific APs which were visible in less than 30% of the measurements (i.e. the first three classes) during 2.5 hours and compared their visibility throughout the 5 and 16 day periods. We found that the visibility of APs might change significantly and inconsistently between periods of different lengths. Table 2.12 lists a few examples. For instance, the AP located in room IFW D31 is visible in 28% of measurements during 2.5 hours, while after 5 days its visibility increased to 57% of measurements. AP location AP visibility during AP visibility during 5 AP visibility during hours period days period days period IFW D31 28% 57% 45% IFW B % 44% 36% IFW A25 75% 40% 44% IFW C31 53% 14% 24% Table 2.12: Variable AP visibility over time as seen by a mobile device in room IFW A44. Changes exist in the other direction as well. APs that have medium-high visibility during 2.5 hours period can have lower visibility as time passes. For example, the AP located in room IFW C31 is visible in 53% of measurements during 2.5 hours, while after 5 days it is visible in only 14% of measurements (Table 2.12). Localization algorithms using filtering techniques based on AP visibility may give such an AP a substantially different weight if their training data is collected over a few hours or a few days, which might cause them to discard the AP altogether. We also found that highly visible APs, which are seen in 90%-100% of measurements, almost do not change their visibility as the period of measurement increases. Examples are given in Table Intuitively, if an AP is visible in more than 90% of measurements during a few hours it is unlikely a coincidence. Most probably this AP is dominant and will continue to appear over time. AP location AP visibility during AP visibility during 5 AP visibility during hours period days period days period IFW A34 100% 99% 99% IFW B42 100% 99% 99% IFW A36 100% 97% 97% IFW C42 93% 90% 91% IFW A32 90% 89% 89% IFW A % 99% 99% Table 2.13: Stable AP visibility over time as seen by a mobile device at room IFW A44. These observations show that AP visibility changes over time and depends on the length of

35 WIFI SIGNAL STUDY measurement period. This should be taken into consideration when developing algorithms that weigh or filter APs based on their visibility APs Signals Correlation Several localization algorithms work under the assumption that WiFi signals from different APs are independent [6, 8]. In the following we check for correlation between signals from different APs at the same location. The most widely used [25] statistical measure of dependence between two variables is their Pearson correlation coefficient. This is calculated by dividing the covariance of the two variables by the product of their standard deviations: corr (X, Y ) = cov(x,y ) σ X σ Y. The Pearson correlation coefficient is a number between -1 and 1. The stronger the correlation between the variables, the closer the correlation coefficient is to either -1 or 1. If the variables do not depend on each other, the correlation coefficient is 0. We follow the classification by Kaemarungsi [17]: the dependence is said to be high if the absolute value of Pearson s coefficient is greater than 0.5, medium if it is between 0.3 and 0.5, low if between 0.3 and 0.1, and no dependence between WiFi signals coming from two APs if less than 0.1. We calculated the correlation coefficient for every pair of APs in every location for both fixed and moving receivers. To increase the statistical validity of our results we consider only APs which have been seen at least 100 times at a particular location. Results are shown in Figure 2.31 for fixed receivers and in Figure 2.32 for moving receivers. Figure 2.31: Correlation class distribution for signals coming from two APs as measured by fixed receivers. Figure 2.32: Correlation class distribution for signals coming from two APs as measured by moving receivers. As we can see, for fixed receivers there are no AP pairs in the high class, meaning there are no AP pairs whose signals are strongly correlated. Almost all AP pairs have correlation coefficient value lower than 0.3. For moving receivers, there are similarly not so many AP pairs whose signals are strongly or medium correlated. The higher correlation measured by moving receivers can be explained intuitively. When a mobile device is moved from one place to another its distance simultaneously changes from both APs. This creates a concurrent effect on both signal strengths, which is perceived in the measurements as correlation. The calculation of correlation classes shows that there is little correlation between WiFi signals coming from different APs, which is small enough for us to assume no correlation.

36 CHAPTER 2. IMPROVING REDPIN LOCALIZATION ACCURACY Distance from AP We have discussed a known WiFi signal property in the Statistical Characteristics section - the farther the transmitter is from the receiver the lower the transmitting signal. Here we would like to show a graphical representation of this property, as being measured by the mobile devices. Figure 2.33 and Figure 2.34 show RSSI values measured from two APs at the same day in room IFW D47.1 and room IFW B47.1 respectively. AP with BSSID 0:3:52:1c:31:60 is located next to room IFW D47.1 and AP with BSSID 0:2:2d:8e:31:9d is located next to room IFW B47.1. The rooms are located on different floors D and B with one floor in-between. The graphs show high signal strength (around -40dBm) for receivers that are close to transmitting APs and low signal strength (below -80dBm) for receivers that are far away from the transmitting APs. In addition, signals coming from close by APs are always captured, while signals coming from far APs are only captured from time to time. Figure 2.33: RSSI as measured by a mobile device at room IFW D47.1. Figure 2.34: RSSI as measured by a mobile device at room IFW B47.1.

37 WIFI SIGNAL STUDY Separation of Fingerprints Separation of fingerprints is essential to LF-based systems [17]. As already mentioned, a limited number of APs can be seen at each location. Also every AP has a limited range of transmission. If two locations are far away from each other and no common AP is received in both, it is easy to distinguish between them. The closer two locations are to each other, the more APs are visible in both, and RSSI from common APs have increasingly similar values. We investigated how feasible location separation is for various distributions of RSSI values. Figure 2.35: Separation of fingerprints for two adjacent rooms IFW D41.2 and IFW D43.1 using APs 0:3:52:1c:33:1 and 0:3:52:4d:e7:93. Figure 2.36: Separation of fingerprints for two adjacent rooms IFW D41.2 and IFW D43.1 using APs 0:3:52:1c:33:2 and 0:3:52:1c:62:0. Figure 2.35 shows an example of fingerprints separation for RSSI signals measured during 4 days in two adjacent rooms IFW D41.2 and IFW D43.1. Signals from both rooms are almost overlapping. It would not have been feasible to distinguish between the two locations using only these APs. However, if we consider a different pair of APs as shown in Figure

38 CHAPTER 2. IMPROVING REDPIN LOCALIZATION ACCURACY we can more easily distinguish between the rooms. This goes to show that more than two APs are needed in order to separate between two close by locations. Figure 2.37: Separation of fingerprints for two rooms on the same floor. In the case of two non-adjacent rooms, two APs are typically enough to separate using their RSSI values. For example, Figure 2.37 shows fingerprints separation for two rooms on the same floor IFW D35.1 and IFW D43.1, which are 3 rooms away from each other. In the open space office, separation of fingerprints proved to be problematic even for two locations far away from each other. Figures 2.38 and 2.39 show examples of fingerprints separation for two locations R1.2 and R1.3, for two different pairs of APs. The locations are about 9 meters away from each other. Measurements were collected over a period of 4 days. Figure 2.38: Separation of fingerprints for two locations in the open space office R1.2 and R1.3 using APs 0:1e:2a:58:c:e and 0:f:cc:dc:7b:4c. Figure 2.39: Separation of fingerprints for two locations in the open space office R1.2 and R1.3 using APs 0:1e:2a:58:c:e and 0:6:b1:14:f1:b5.

39 WIFI SIGNAL STUDY Effect of user presence on RSSI WiFi waves are absorbed by various materials including human body, due to its high water content. People therefore affect the propagation of WiFi signals. The effect is intensified because people are constantly moving, coming and leaving, making the environment dynamic and the behavior of wireless signals even less stable. Figure 2.40: Variations in RSSI from a fixed receiver during night, early morning and morning (from 00:00:00 to 11:30:44). Figure 2.40 shows RSSI as measured in one of the rooms of IFW building between midnight and 11:30 in the morning. We can separate the observed RSSI into three periods: from midnight to approximately 6:00 am when the strength of the received signal from all APs is quite stable; from 6:00 am to approximately 9:30 am when the strength of received signal is less stable; and starting from 9:30 am, we observe an erratic behavior of the signal strength. Looking at the RSSI measurements we can deduce the time people start working at the campus. The effect of people s presence on WiFi signals has been studied in the past [17, 20] and our results are similar, confirming their findings Type of Mobile Device During our study we found that different mobile devices in the same room might capture WiFi signals from significantly different set of APs. We have only two of such examples, since in most cases there was only a single user collecting WiFi in a given room. Figures 2.41 and 2.42 show graphs of RSSI values in the small room IFW A44 as captured by two different mobile devices at the same time period during the same day. The devices were in

40 CHAPTER 2. IMPROVING REDPIN LOCALIZATION ACCURACY 29 two different physical locations within the room. Different colors on the graphs indicate RSSI values from different APs. On Figure 2.42 we can see a large number of APs, while on Figure 2.42 there are only a few. Figure 2.41: RSSI values in room IFW A44 as measured by an iphone 3GS. Figure 2.42: RSSI values in room IFW A44 as measured by an iphone 3G. Another such example is shown in Figures 2.43 and 2.44 in room IFW D43.1. Figure 2.43: RSSI values in room IFW D43.1 as measured by an iphone 3GS. Figure 2.44: RSSI values in room IFW D43.1 as measured by an iphone 2G. Both devices which capture less APs (Figure 2.41 and 2.43) are iphone 3GS, while other devices are older iphone models. Indeed, we checked other measurements taken by 3GS and the number of captured APs is always smaller than a number of APs captured by other iphone models. This leads us to believe that the new iphone has degraded WiFi reception. Such differences in receivers capabilities may cause significant problems to localization algorithms. If most of the fingerprints in a database are collected by one type of devices, and they are used by an algorithm to derive the location of devices of another type, the result can be incorrect even when all other conditions are perfect. Hence the type of receiver, its model and even firmware version, are important to consider during the development of localization systems.

41 WIFI SIGNAL STUDY Summary Here we provide a short summary of our WiFi signal study based on a user-contributed data set (Section 2.4). Physical AP may broadcast several virtual APs and the transmitting signal might vary significantly from one virtual AP to another. In addition, not all virtual APs coming from the same physical AP are always captured by the mobile device. WiFi signal strength is lower the farther the receiver is from the AP, due to the path loss propagation property (Sections and 2.5.5). However, if a receiver is closer to one AP than it is to another, the RSSI from the closer AP is not necessarily higher than the RSSI from the farther AP at the same receiver position. (Section 2.5.1). Standard deviation of RSSI is higher for moving receivers than for fixed receivers at the same location from the same AP. The mean of RSSI does not vary by much between fixed and moving receivers, unless they are close to the transmitting APs (Section 2.5.1). In most cases, the lower the signal strength, or the farther the receiver is from the AP, the smaller the standard deviation of RSSI and so the fluctuations. However this is not always the case for moving receivers. (Section 2.5.1). Distribution of RSSI might vary significantly with the time of day, duration of measurement, and over time (Section 2.5.2). The number of APs seen by a mobile device at a certain location is not constant over time. It is not static even when there are no environmental changes, due to the constant change in the number of virtual APs broadcasted by physical APs (Section 2.5.3). The number of APs with lower visibility captured by a mobile device increases over time (Section 2.5.3). The visibility of APs might change significantly and inconsistently. However, highly visible APs, which are seen in 90%-100% of measurements, almost do not change their visibility as the period of measurement increases (Section 2.5.3). There is but a little correlation between WiFi signals coming from different APs (Section 2.5.4). Typically it is enough to have two APs for distinguishing between two locations which are far away from each. However, more APs are needed in order to separate between two close by locations or between locations with no separating walls (Section 2.5.4). Presence of people has a significant effect on RSSI (Sections and 2.5.7). Receivers of different models and firmware versions have varying WiFi reception capabilities (Section 2.5.8).

42 CHAPTER 2. IMPROVING REDPIN LOCALIZATION ACCURACY Conclusion Our WiFi study shows how unstable WiFi signal can be in dynamic environment and its dependency on many unpredictable factors. Behavior of WiFi signal changes significantly when measured over different periods and frequency, and we did not find any strict pattern. Many localization algorithms use different techniques to improve the locator accuracy based on specific data samples and show good results. Our insights led us to abandon the route of data-specific improvements such as filtering, which is prone to over-fitting and loss of generality. 2.6 Evaluation of Localization Algorithms We evaluated the accuracy of the Redpin, Bayesian, and SVM methods in different setups using an evaluation tool we developed. We then compared Redpin and SVM in terms of performance Data Set To evaluate the algorithms we use a data set collected by users during 6 weeks of WiFi measurements (described in details in Section 2.4). Users were collecting data in free form in different locations, typically their workplaces. During the first 3 weeks fingerprints were collected from fixed receivers, with users leaving their mobile device in one spot. During the last 3 weeks fingerprints were collected from moving receivers, with users moving their mobile device between different spots in the same location. In total unique measurements were collected. We studied and analyzed the measurements from both fixed and moving receivers to better understand the behavior and characteristics of WiFi signals as collected by users (Section 2.5) Evaluation Tool The most popular approach for estimating the accuracy of a given classifier is running it through Cross Validation [26]. Cross validation involves repeatedly partitioning a given dataset into non-overlapping training set and testing set. The training set is being used to induce the classifier, which is then validated using the unseen instances in the testing set. To evaluate the accuracies of different indoor localization algorithms over different setups we developed a generic data evaluation tool. Our tool performs k-fold cross validation of a localization algorithm given as input over a collection of fingerprints. The main strength of the tool is that it makes it possible to perform cross validation of any inducer over a collection of data points of arbitrary type. Figure 2.45 describes the data model of the tool. CrossValidator implements the IModelEvaluator interface and evaluates the provided inducer over the given set of data by performing k-fold cross validation. Specifically, dataset D is randomly divided into k distinctive and exhaustive sets, D 1, D 2,..., D k of approximately equal size. The inducer which implements the IInducer interface is trained and tested k times; in each iteration i it is trained over D/D i and tested on D i. ILocatorInducer is an interface that extends IInducer but knows how to

43 EVALUATION OF LOCALIZATION ALGORITHMS Figure 2.45: Data Evaluation Tool class diagram. work with the Fingerprint and Location data types provided by the Redpin system. LocatorInducer is a specialized implementation of ILocatorInducer and IExperimentLocator which uses a locator algorithm as the inducer function. The IExperimentData interface adds a level of indirection between the underlying localization algorithm and the data using the setup method. This method allows filtering out portion of the data from being seen by the algorithm, keeping it as covert set. ILocator is an interface provided by the Redpin system. Any locator which implements the ILocator interface can be evaluated using the tool. We used the existing RedpinLocator (Section 2.3.1) and implemented two other locators BayesianLocator and SVMLocator (Section 2.3.3). BayesianLocator is a general locator based on the Bayesian theorem, which may operate under the assumption of any given distribution of readings. For instance, we assume Normal distribution of readings [6] and use BayesianLocator as follows: new BayesianLocator<ReadingDistributionNormal>(); The histogram of readings as suggested by X. Chai and Q. Yang [8] is also implemented and can be used instead: new BayesianLocator<ReadingDistributionHistogram>(); For every data set which we evaluated, histogram of readings provides lower or equal results

44 CHAPTER 2. IMPROVING REDPIN LOCALIZATION ACCURACY 33 as normal distribution of readings. Other works have also discovered this superiority of using the normal distribution over an histogram [12]. We thus do not present results corresponding to the former approach. From now on we use the term Bayesian locator or Bayesian method to refer to the Bayesian locator assuming a normal distribution of readings as described in Section MeasurementClassifier which implements the IClassifier interface is used to predict a location of a measurement from the testing set. To calculate the accuracy of a specific algorithm, we compare the predicted location with the actual, expected one. Using the tool simplifies the evaluation of location detection algorithms over different data sets. Following is a Java code excerpt showing how to use the system: // create new inducer LocatorInducer locatorinducer = new LocatorInducer(); locatorinducer.setlocator(locator);//svmlocator(), RedpinLocator() //BayesianLocator<T>(), or any other // create new cross validator CrossValidator<Fingerprint, Location> crossvalidator = new CrossValidator<Fingerprint, Location>(); crossvalidator.setinducer(locatorinducer); crossvalidator.setdataset(dataset); // evaluate the model and get the accuracy crossvalidator.evaluate(); double accuracy = crossvalidator.getaccuracy(); To increase confidence in the evaluation results we repeated the k-fold cross validation 100 times. We used k=10 for large data sets (at least 1000 fingerprints) and k equals to size of a dataset for small datasets (leave-all-but-one) Accuracy The accuracy is defined as the portion of measurements the algorithm correctly localizes. Formally, let δ (A, m) be 1 if algorithm A localizes measurement m correctly, 0 otherwise. The estimated accuracy is then acc = 1 n k i=1 m j D i δ (A, m j ) Where n is the total number of elements in the dataset D. We average the accuracy over 100 runs Results In the following we present the evaluation results for different setups. Short term measurements We start by evaluating the algorithms over a portion of our data, corresponding to a short time period of a few hours. We took measurements from each location in IFW and RZ, and

45 EVALUATION OF LOCALIZATION ALGORITHMS from both fixed and moving receivers. Ten measurements were taken from each location in order to increase the statistical validity. These measurements were recorded during the first hours of the Wifi Signal Study. Leave-all-but-one cross validation was used for evaluating the algorithms. Table 2.14 presents the resulting accuracy for all algorithms over four different data sets: 1) measurements from locations in IFW taken by fixed receivers; 2) measurements from locations in IFW and RZ taken by fixed receivers; 3) measurements from locations in IFW taken by moving receivers; 4) measurements from locations in IFW and RZ taken by moving receivers. Figure 2.46 is a graphical view of the same results. Bayes Redpin SVM Fixed receiver IFW Fixed receiver IFW & RZ Moving receiver IFW Moving receiver IFW & RZ Table 2.14: Accuracy of the algorithms for the short term setup accuracy Fixed receiver [IFW] Fixed receiver [IFW & RZ] Moving receiver [IFW] Moving receiver [IFW & RZ] Bayes Redpin SVM Figure 2.46: Accuracy of the algorithms for the short term setup. It can be seen that all algorithms, in every location, perform better on measurements gathered by fixed receivers, than on measurements gathered by moving receivers. This goes in line with the aforementioned signal study discussion, which stated that readings from fixed receivers provide better conditions for location detection algorithms since it is easier to perform fingerprint separation. However it is not realistic - users do not query for their locations from

46 CHAPTER 2. IMPROVING REDPIN LOCALIZATION ACCURACY 35 fixed spots. The good results of both Redpin and SVM can be explained by the fact that all measurements are close to each other in time, and hence are quite similar. Both algorithms use deterministic approaches comparing a target measurement with existing measurements in the database, picking the location of similar measurements. Both Redpin and SVM compare RSSI values directly, as opposed to the Bayes approach of first inferring a distribution from the values and use this distribution to classify. The strength of the Bayes approach lies in its generalization capabilities, which are less important with a small input set of mostly similar elements. Bayes accuracy is low, since the small number of measurements per location, and their proximity in time and values, do not allow it to infer an accurate Normal distribution. Another point to note is that in all cases, the addition of locations from the RZ building increases algorithms accuracy for both fixed and moving receivers. Intuitively, additional measurements from easily distinguishable locations (such as different buildings) increase accuracy. In the following we will focus at the more challenging part of the problem, i.e. only locations in a single building, the IFW building, and using moving receives. Long term measurements The accuracies of the algorithms over the whole dataset collected by moving receivers are shown in Table Bayes Redpin SVM Accuracy Table 2.15: Accuracy over the entire dataset collected by moving receivers. Bayes accuracy is considerably improved comparing to the shorter term evaluation, but it is still lower than the other two algorithms. Both Redpin and SVM show better results on the larger dataset, with SVM performing better than Redpin. However it is not trivial to use large database (more than fingerprints) with deterministic algorithms, since comparing a target measurement with all fingerprints in the database takes prohibitive amount of time. For example, the average query time for the Redpin algorithm over a dataset of 6000 fingerprints is more than 10 seconds (Section 2.6.4). It would therefor be beneficial to know what accuracies the algorithms can achieve over datasets of smaller size. Instant Labeling In the following we compare the accuracy of algorithms over different sizes of datasets, consisting of measurements collected by a simulated instant labeling technique. To do this we created sets of increasing size from 10 to 200 randomly chosen fingerprints per location. We consider only locations where at least 200 fingerprints were collected. Randomly selecting fingerprints simulates instant labeling, since the probability of getting two consecutive 1 fingerprints in time (within 30 seconds of each other) is about and the probability of getting a longer streak of consecutive fingerprints is even smaller. We used leave-all-but-one cross validation for evaluating the accuracy over these sets, shown in Figure 2.47.

47 EVALUATION OF LOCALIZATION ALGORITHMS accuracy Bayes Redpin SVM # of measurements per location Figure 2.47: Accuracy when instant labeling technique is used to collect fingerprints. Similarly to what we have seen earlier, the Bayesian approach shows the lowest accuracy and SVM the highest. It is important to mention that both Redpin and SVM do not improve by much after 30 fingerprints per location, while the Bayesian method reaches stagnation only after 110 fingerprints per location. This demonstrates that the required dataset size depends upon the specific algorithm being used. The higher accuracy of all algorithms compared to other works [22] is explained by the nature of our dataset. Most of the fingerprints in it are collected from non-adjacent rooms, and that allows the separation of fingerprints to be done more accurately than for adjacent rooms with similar fingerprints. The results are summarized in Appendix A.1. Interval Labeling In the next experiment we compare the accuracy over datasets of different size collected using interval labeling. We created the datasets by choosing a single random interval of fingerprints per location, varying from 5 to 100 minutes in length. Since measurements were taken by the mobile device every 30 seconds, a 5 minutes interval is equivalent to 10 measurements per location, and a one hour interval is equivalent to 120 measurements per location. We used leave-all-but-one cross validation for evaluating the accuracy over these sets, shown in Figure The y-axis scale begins with 0.4 to show the curves in more details. The Bayesian method keeps showing the lowest accuracy and SVM the highest - it reaches almost 100% accuracy. Both Redpin and SVM do not improve much after 15 minutes interval per location (equivalent to 30 measurements using instant labeling), while for the Bayesian

48 CHAPTER 2. IMPROVING REDPIN LOCALIZATION ACCURACY Bayes Redpin SVM accuracy interval in minutes Figure 2.48: Accuracy of the algorithms when interval labeling is used to collect fingerprints. method it happens only after 40 minutes intervals. The curves are not completely smooth due to to the occasional low number of intervals. We do not have many intervals that last more than minutes since users were not collecting data continuously for so long due to interruptions such as phone calls, or leaving the location. The results are summarized in Appendix A.2. Interval vs. Instant Labeling In the following we show a different view of the same accuracy graphs from the previous two sections, plotting both the instant and interval results per algorithm. Figure 2.49 shows a comparison between interval and instant labeling for the Bayesian method, Figure for the Redpin algorithm, and Figure for the algorithm based on SVM. Our take here is that the interval labeling technique performs better for all algorithms comparing to the instant labeling technique. So far we used datasets of the same size, consisting of either N random measurements or a single random interval spanning N measurements. Intuitively, using N intervals is better than using one, and superior to using N measurements, simply because by definition there will be a higher number of measurements. These measurements would also be similar to one another. We would like to verify this hypothesis nevertheless. In the following we compare interval labeling using N random intervals of different length, to instant labeling using N random measurements. This reflects the reality where instead of taking a single measurement when users contribute their location, the system continues to take measurements from the same location in background.

49 EVALUATION OF LOCALIZATION ALGORITHMS Bayes accuracy interval instant # of measurements Figure 2.49: Accuracy of the Bayesian method using instant and interval labeling. Redpin accuracy interval instant # of measurements Figure 2.50: Accuracy of Redpin using instant and interval labeling. Figure 2.52 shows the results for Redpin. A table summarizing the results can be found in Appendix B. It can be seen that even using intervals of only 5 minutes improves accuracy compared to instant labeling. Using intervals of 10 minutes improves accuracy from using 5 minutes intervals by 1.6% on average, 15 minutes improves from 10 minutes by almost 4% on average. From 15 minutes onwards the improvements are small, for example going from 20 to 25 minutes improves accuracy by only 0.17%. This confirms our previous observation, that interval labeling hits diminishing returns for intervals longer than 15 minutes.

50 CHAPTER 2. IMPROVING REDPIN LOCALIZATION ACCURACY 39 SVM 1 accuracy interval instant # of measurements Figure 2.51: Accuracy of the algorithm based on SVM using instant and interval labeling. Instant vs. Interval 0.98 accuracy min 10 min 15 min 20 min 25 min 30 min instant number of samples per location Figure 2.52: Accuracy of the Redpin using N measurements and N intervals. Performance of SVM vs. Redpin All results show that the algorithm based on SVM performs better, so we would like to replace Redpin with SVM for the locator of the Redpin system. On one hand, it could take considerable time for SVM to create a prediction model using all fingerprints in the database, but on the other hand predictions based on the model should be fast. In contrast, Redpin does not have a training phase, but matching all fingerprints in the database against a target could take long for large databases. In the following we compare the performance of SVM and Redpin in terms of both query

51 EVALUATION OF LOCALIZATION ALGORITHMS time and training time. We used LIBSVM which has two implementations in C and in Java. It was also interesting to compare these two implementations which we dubbed SVM C and SVM Java. We created datasets of between 1000 and 6000 random fingerprints from the data collected during WiFi Signal Study and evaluated Redpin, SVM C, and SVM Java on every fingerprint in a set, using the rest as training data. We then calculated the average performance in terms of training and query times. Results are shown in Tables 2.17 and Figure 2.53 shows comparison of query time and Figure 2.54 comparison of training time. Number of samples Redpin SVM C SVM Java Table 2.16: Average query time in milliseconds. Number of samples Redpin SVM C SVM Java Table 2.17: Average training time in milliseconds. Redpin requires significantly more time to perform a query, while SVM Java shows the best performance for average query time. SVM C takes more time than SVM Java to answer a query due to the additional overhead incurred by the interface between the C code of LIBSVM and the Java code of the Redpin system. It is evident that the average query performance for all algorithms grows linearly with a number of samples in a dataset. As already mentioned, Redpin does not have a training phase. The average training time for SVM Java is the worst of the batch, as can be observed on Figure The time complexity of its learning component is exponential, as opposed to SVM C whose average training time grows linearly with a number of samples in a dataset. The combined performance of SVM is higher, but without training time it takes significantly less time to answer a location detection query. Also, the average query time of Redpin is prohibitive even for datasets of as small as 2000 fingerprints, for which it takes more than 2.5 seconds to answer a query. Luckily, the SVM prediction model does not have to be rebuilt every time a new fingerprint is added to a database. For large databases, it is unlikely that a single fingerprint adds much information to the model. Therefore, we decided to adopt the following approach. SVM prediction model shall be built periodically (for example, every hour) in the background,

52 CHAPTER 2. IMPROVING REDPIN LOCALIZATION ACCURACY 41 query time Redpin SVM C SVM Java milliseconds # of samples Figure 2.53: Average query time of Redpin, SVM C, and SVM Java for datasets of increasing size. training time Redpin SVM C SVM Java milliseconds # of samples Figure 2.54: Average training time of Redpin, SVM C, and SVM Java for datasets of increasing size. and will replace the old prediction model once done. All location queries sent to the locator will be answered using the prediction model available at that time. Using this approach we

53 REDPIN IMPROVEMENTS solve the performance problem of the current Redpin algorithm and increase accuracy of the locator. However, some of the users queries might not be answered using the most recent fingerprints. Conclusion Our results show that the localization algorithm based on SVM is more accurate and, using the optimization technique described above, is also better in terms of performance. In addition, the interval labeling approach increases the accuracy of the algorithms. Databases of 30 measurements per location are sufficient to provide reasonable accuracy. Requiring a certain number of measurements per location has been discussed in the literature [2]. An alternative approach is to put a requirement over the total number of measurements, regardless of location [22]. For crowdsourcing-based systems where data collected in a nonsystematic way, it does not suffice to rely on the dataset size alone, because it might be the case that we have a large number of fingerprints from one location, and almost none for the other. We therefore stick to requiring a certain number of fingerprints per location. The overall database can still be large if there are many locations. 2.7 Redpin Improvements We implemented the SVM-based localization algorithm for the Redpin locator (Section 2.7.1) on the server side, and interval labeling on the existing Redpin iphone client application (Section 2.7.2) SVM on Redpin Server To implement the periodic creation of SVM models we use a scheduler TrainSVMTimerTask which extends basic java.util.timertask Java library class for thread related tasks. TrainSVM- TimerTask starts when the Redpin server starts, performs SVM training, reschedules itself for a predefined time period (for example, one hour), and goes to sleep. Figure 2.55 visualizes the way that the SVM prediction model is built. SVMSupport class is responsible for operations such as train and predict. When train is called it uses CategorizerFactory to generate labels for locations (item classes) and for BSSIDs which are used as attribute identifiers. All location-tagged measurements are converted into the SVM input format 2 defined by LIBSVM and svm scale.run() and svm train.run() of the LIBSVM library are called. These classes build the SVM model based on the formatted data received as input. Scaling is important to avoid numerical difficulties during calculation. Indeed, we checked that without scaling SVM provides worse results. The final step of the process is replacing the old model with the new one. To predict a location given a target measurement, SVMSupport converts the measurement into LIBSVM s format and invokes svm predict.run() which derives the class (location) of the target measurement based on the SVM prediction model received as input. SVM training uses LIBSVM C module and prediction uses LIBSVM Java module. 2 A Practical Guide to Support Vector Classification: cjlin/ papers/guide/guide.pdf

54 CHAPTER 2. IMPROVING REDPIN LOCALIZATION ACCURACY 43 TrainSVMTimerTask SVMSupport CategorizerFactory MeasurementHome svm_scale svm_train train buildcategories sleep getallmeasurements measurements transformtosvmformat run run replacethemodel Figure 2.55: SVM train sequence diagram. LIBSVM classes are marked in gray Interval Labeling on Redpin Client Interval Labeling for the Redpin Client application is implemented as follows. When a user adds a new location, the location is stored as the current location, and an instance of IntervalScanner class is instantiated. IntervalScanner runs in the background, and periodically (e.g. very 30 seconds) calls the Sniffer which performs a scan of WiFi networks and generates a WiFi measurement. The Sniffer notifies the IntervalScanner upon finishing the scan. The latter proceeds by creating a new fingerprint, i.e. attaches the current location to the new measurement. The fingerprint is sent to the server and is stored there. IntervalScanner stops scanning whenever it detects a significant movement through the built-in iphone accelerometer. Scanning is also stopped whenever users are adding a new location or try to locate themselves. When IntervalScanner stops scanning it notifies its delegator and the latter releases the IntervalScanner process Additional Improvements While working with the Redpin system we found a few points for improvements in the current system. Firstly, Redpin locator introduced memory leaks when working with large datasets (about 2000 fingerprints). This was due to the large number of unclosed database resource handles. Another problem was with using a non-normalized database schema which did not allow performing operations such as join while querying the database. We transformed the schema to 3NF form ensuring database structure is optimized for querying, insertions, deletions, and updates. We also introduced transactions to insert and delete data ensuring atomicity of the database

55 REDPIN IMPROVEMENTS and keeping it consistent and complete for cases of system failure. This eliminated partial insertion of a fingerprint into the database which was allowed before. Lastly, we replaced nested loading of objects from the database which required tens of SQL queries to the database, with a single query using the join operator.

56 3 User Motivation Study 3.1 Related Work Crowdsourcing assumes creating an open system through which users can participate and contribute fingerprints. There are several works in the field which study users experience [6, 23], try to motivate users, and to minimize the user s effort in contributing fingerprints [3, 10]. For example, Organic Indoor Location Discovery [10] presents a graphical user interface to capture tagged locations with little user effort. Another work by A. Barry, B. Fisher, and M. L. Chang [3] presents an indoor wireless localization system based on crowdsourced data collected from end-users. In order to motivate users they implemented Friend-Finding Service which shows users location on an old magical map. End-users can publish their location on the map as well as to search for their friends. Another work [23] analyses the Active Bat [13] indoor positioning system from the perspective of people needs and users motivation. They suggest classifying users into groups by use case, study factors that may motivate different groups of people, and use this information to develop applications targeting specific needs. We have collected and classified various ideas and incentives that can motivate users to contribute in this part of the work. We decided to choose two different classes of incentives to motivate users and for each implemented an application to analyze real-life usage. One is the Hunt The Fox game, and the other is a social twitter-like network called Meex. Both applications have been implemented on the iphone. We conducted a user study for Hunt The Fox followed by an analysis. This second part of the work is structured as follows. In the next section we go over different incentives that can motivate users to contribute. Section 3.3 describes our novel Hunt The Fox game with its analysis and user study. In Section 3.4 we present the social application Meex and conclude in Section

57 INCENTIVES 3.2 Incentives We gathered different ideas for motivating users to contribute their location, and existing implementations of some of them. The incentives can be categorized into four classes as shown in Table 3.1. Incentive category Ideas and existing implementations Games Ownership. The idea is to grant users virtual ownership of the location when they submit their location. Users are required to continuously submit their location in order to maintain ownership of it, which helps motivating in the long term. Map discovery. The idea is similar to the Warcraft 1 game where a map is hidden and users have to uncover the map step by step by submitting their locations. Hunt The Fox, developed in the course of this study and is described in details in Section 3.3. Social networks Users tracking. The idea is to track others in the user s vicinity. As soon as users assign tags to their current location, they can see other users who have visited or are visiting the same location. Sharing user s current location on Twitter or another existing social network. Google latitude 2 - a social network having a mobile phone client which allows users to see the physical outdoor location of their friends and share their own location. The same idea can be applied or combined with indoor locations. Foursquare 3 - a location-based social network with an iphone client that allows users to update their symbolic location (venues) and connect with their friends using text messaging. Meex, developed in the course of this study and is described in details in Section

58 CHAPTER 3. USER MOTIVATION STUDY 47 Incentive category Ideas and existing implementations Utilities Friend-Finding Service [10] Creating a floor plan of a building. The idea is to allow users to create a plan of their floor or building. Changing mobile profiles base on location within the office. The idea is to allow users to teach their mobile phone about the different locations in the office so that it can change its profile ( silent, meeting, and so on) according to the user s location. Redpin indoor localization system 4 and other Where am I? -like applications. Other incentives User s good will, curiosity. Users are paid for their contributions. Payment can be: points, free internet time, actual money. Credit based - for example: users are given access to an application after they contribute 10 tags. Table 3.1: Incentives for motivating users to contribute their location. 3.3 Hunt The Fox In order to motivate users to contribute we developed a game called Hunt The Fox. It is initially aimed at people who are familiar with each other, for example people who work or study together and play the game in their spare time. Hunt The Fox is a group oriented game, but it is possible to play it by a single person as well. The idea behind the game is to show players the floor plan of their surroundings, with a fox icon in one of the rooms. The player needs to go physically to that location and press the catch button in order to catch the fox and receive points. Afterwards, the fox appears in a different location and players need to continuously chase the fox to achieve higher scores Architecture Overview Similarly to Redpin, Hunt The Fox utilizes a client-server architecture using an iphone application as a client. Every 60 seconds the client requests the up-to-date location of the fox 4

59 HUNT THE FOX from the server, and redraws the fox in a new position on the map according to the (x,y) coordinates received. On the server side, every (x,y) coordinate is associated with a location. When players click catch on their clients, the client measures the WiFi signal strength of all APs in the range of the mobile device, and sends them to the server. Along with the signal measurement, the client sends the identifier of the room where the fox is drawn, assuming the user is there. The server then localizes the client by the received fingerprint. The calculated client s location is compared to the actual fox location. In case of a match, the fingerprint is considered correct and is stored in a database for use in future localization computations. Otherwise, the fingerprint is considered suspicious and the client receives a cheating warning. The fox position changes when one of the players catches the fox. The fox is caught when the following two conditions are true: the client s fingerprint is correct and there is no other player who has already caught the fox. The server changes the location of the fox every 5 minutes if no one catches it. The next location of the fox is chosen randomly from all possible locations but the current one. To calculate locations on the server side we use a localization algorithm based on support vector machine (Section 2.3.3). Our experiment has shown that at least 5 fingerprints from every location are needed to get accurate location predictions. For that reason we consider the first 5 fingerprints for every location as correct. Detection of incorrect data from clients during this initial phase, i.e. users cheating about their location, is beyond the scope of this work. Communication between the server and client is done through asynchronous JSON messaging using a polling mechanism. The implementation of the data model is based on the one from Redpin for both the client and server sides. The server is implemented in Java and thus can run on any operating system. The database used is SQLite, but any other database having Java Database Connectivity (JDBC) API can be used instead. iphone Client The iphone client user interface is implemented in Objective-C. It has been designed to be as friendly, simple, and intuitive as possible. It has three screens: the start screen, the play screen, and the scores screen shown in Figures The start screen (Figure 3.1) is where players choose their nick name, and select one of predefined floor plans as the hunting zone. The play screen (Figure 3.2) is where the game takes place. It shows the fox in one of the rooms (locations) in the selected floor plan, according to the coordinates sent by the server. The scores screen (Figure 3.3) lists the nick names of the players along with their scores. We chose to display the fox location on a floor-plan following the principle of less words and more graphics to make it more intuitive to the user. As the game is aimed to be played by people in familiar environments like their office or a private apartment, we believe it would be easier for them to understand where the fox is using a graphical map. To make the game more group oriented, all players on the same floor-plan go after the same fox. If two players in the same room press catch, only the quickest one will actually catch the fox and get a point, while the second player will receive a message saying the other player has already caught the fox (Figure 3.4). We believe that playing together get users more motivated as they have to compete with each other in achieving the highest score. We are aware that users care about their privacy, and thus Hunt The Fox does not save device identifiers or any other personal information.

60 CHAPTER 3. USER MOTIVATION STUDY 49 Figure 3.1: Start screen. Players can choose a nick name and select one of the predefined floor plans so called hunting zone. Figure 3.2: Play screen. The floor plan selected by the player with the fox randomly placed in one of the rooms. Figure 3.3: Scores screen. List of the players and their scores. Figure 3.4: Cheating message.

61 HUNT THE FOX User Study To investigate whether Hunt The Fox entices people to contribute, we conducted a user study. The study examined whether users play the game at all, how often they play, when they play and how. In addition, we wanted to know how users motivation evolved with time and whether additional incentives (like a prize) increase their motivation. Overview We recruited 9 participants from two groups, similarly to the setup of the MultiM study: 4 users from the office group and 5 users from the campus group. All participants installed Hunt The Fox on their iphones and got instructions of how to play the game, explicitly asking them not to cheat, i.e. not to press catch button when they are not in the same room as the fox. The study lasted four weeks. The first week was mostly spent trying to get all participants to install the game on their iphones. For every week, starting the second week, a winner (the player who achieved the highest score) was announced and all scores were reset. In addition, the winner got a small prize. We did not tell the participants about the prize upfront but rather only after the second week, since we wanted to compare how people play when there is a prize versus not having one. Scores were reset every week so that users will not lose hope of being in the first place, in face of one strong competitor who dominates the scoring table. To remind users about the game we were sending them an with up-to-date scores every other day. At the end of the study we asked all participants to fill out a short survey to better understand their usage experience and incentives. The survey and users answers can be found in Appendix C. Results Users caught the fox a total of 224 times, and 248 measurements were collected. This means that 24 times a user pressed the catch button, the fingerprint was correct, but another user has already caught the fox. This is a good indicator that people do play together and the group oriented approach we have chosen works. During the first week users caught the fox 41 times, during the second week - 89 times, during the third week - 45 times, and during the last week - 48 times. All participants were playing the game during the first two weeks. In the third week only 4 participants played - two from each of the office and campus groups, the two other office users were out of office. Five participants played the game during the fourth and last week - four from the office group and one from the campus group. Seven users filled in the survey. One user gave us a feedback that he did not have a chance to play the game, and one chose not to give any feedback. Three users played Hunt The Fox twice a week, 2 users - three times a week, one played once a week and one less than once a week. From the survey results we learned that most of the users played the game by actively chasing the fox. However there was one user who was waiting for the fox to come to the user s place; and one user who used both approaches to catch the fox. Figure 3.5 shows how many new measurements were added as a function of the day into study. We can observe that users played almost every day except for weekends (days 3-4, 10-11, 17-18, and 24-25) and public holidays (days 23 and 26). During the first two weeks users were more active than during the following two weeks. However the 22nd day shows high

62 CHAPTER 3. USER MOTIVATION STUDY 51 Figure 3.5: Number of new measurements added as a function of the day into study. activity when two participants from the office group were playing at the same time, probably because they had more time to play the game just before the public holidays. containing scores was sent on the morning of days 6, 8, 13, 15, 20, 22, and 28 (the last ) which explains the increased activity of users during these days compared to the day before. The announcing the winner was sent on days 13, 20, and 28. Conclusion It seems that the game succeeds in motivating people to contribute. Even after the study was over we had users continuing to play Hunt The Fox. The game was more popular among participants from the office group (they caught the fox in total 183 times) than among participants from the campus group (they caught the fox in total 41 times). The popularity of the game in the office group was so high that watchers from the side have asked to participate in the study. We attribute this success to the office setup, where all users sit together in a big open space, can communicate with each other easily and see when other people in the office are playing the game. In contrast, participants from the campus group are in separated rooms, they can not see each other nor communicate with each other so easily. Moreover, the office people had access to every room where the fox appeared and could follow the fox wherever it went, while people in the campus could not enter certain rooms such as server rooms or a meeting room during a meeting. Therefore they had to wait 5 minutes until the fox changed its place after not being caught. This disrupts the game flow and, most likely, players lose their interest. Two users from the campus group pointed out this problem.

63 MEEX As the survey results show, adding the additional prize incentive seems to increase users motivation - all users but one were motivated after learning about the prize for the winner. However, according to actual results only one user was playing more after learning about the prize. According to the survey, the main reason people did not play the game is because they forget about it, which is understandable since everybody were playing during their working day. One user pointed out that there is no benefit to him by playing the game, and one complained about the game being cumbersome. The with scores sent to the participants did not only remind them about the game, but also increased their motivation. One user even asked for more frequent updates. This is a good indication for us that we chose the right balance for these updates, as we were afraid in the beginning to spam users with s and decrease their motivation to play. Even though the game motivates users, this motivation does not persist in the long run, as we learn from both the answers to the survey and actual results. No one answered yes to the question would you still play after 6 or so month, and only two users answered probably. In addition, the number of measurements collected per week was decreasing with time. The increase in the number of measurements from the first week to the second week is merely because all participants installed Hunt The Fox on their iphones towards the end of the first week. Possible Improvements One possible improvement is to allow the users themselves to specify in which rooms they would like to play. This should solve the problem of non-accessible rooms and enable continuous flow of the game. Another improvement is to provide the users with clearer instructions. For example, it was not obvious to everyone that the fox changes its location if nobody catches it within five minutes. Each user gave us some feedback of how to improve Hunt The Fox. One of the suggestions was to put the fox in more places within the same room and make it run from one place to another in an animated manner. Another suggestion was to issue an alert whenever the high score changed, to motivate competition. One user suggested improvements to make Hunt The Fox more group oriented by requiring collaboration between several people to catch the fox. For instance, two players drive the fox into a specific direction where a third player awaits to catch it. Another interesting suggestion was to place a few foxes simultaneously at different places and requiring that the player catches several foxes before another batch of foxes appears again. One other suggestion was to have the application consider the fox catched whenever the user enters the room where it is located, without the need for pressing the catch button. All suggestions we received are well thought after and are worth considering. We are certain that after implementing a few of these ideas, Hunt The Fox could receive more attention and provide motivation for people to contribute their location. 3.4 Meex To keep users motivated for a longer period of time we developed an application called Meex with different characteristics. Meex is a social application which allows commu-

64 CHAPTER 3. USER MOTIVATION STUDY 53 nication between people who share the same physical location. Meex is aimed at people who initially do not know each other. The name comes from a composition of meet and mix. Meex allows users to virtually check-in to their current location and find other users who are checked-in and willing to communicate by posting messages on a chat board. It is usually difficult for people who do not know each other to communicate with each other, while chatting via a social application like Meex is simpler, because they can stay anonymous. In case they find common interests while chatting, they can actually meet because they are in the same physical area. This is the meet part. Mix is about mixing different people in different situations with various interests Use Case Meex applies to many different use cases, and we chose one to better demonstrate the typical usage and aim of Meex. A student is working on a homework exercise in the computer science building and having difficulties or needs to discuss a certain point with somebody. It would not be easy, both physically and mentally, for the student to go and ask around the building to see whether somebody else is working on the same exercise and willing to discuss it. However, it could be easy using Meex. The only thing the student has to do is to check-in into his/her current location and leave a message on the board. This way Meex does not only help to solve the student s problem, but maybe also to find a new friend. Meex is taking WiFi measurements in the background, associating them with the student s location and this way it is learning about the environment Architecture Overview Similarly to Hunt The Fox, Meex is a client-server architecture using iphone as the client. Users check-in to their current location and view posts from the other users in the same location via a messaging board. Users can post their own messages, which are visible only to users in their vicinity. In this way, Meex is clustering users according to their physical location. When users perform the check-in operation on their client, Meex starts interval scanning, meaning every 30 seconds the WiFi signal strength from all APs that cover the area around the mobile device are scanned and sent to a server along with the current location of the user (checked-in location). The interval scanning continues until one of the following occurs: the user checks-in to another location, the user exits the application, or continuous movement is detected, signifying a possible change of location. Movement detection is done using the built-in iphone accelerometer. If interval scanning is stopped, it will be resumed once the user posts a new message or refreshes the messaging board - in the both cases we assume that the user is still in the checked-in location and restart scanning for APs. The server stores fingerprints received from clients in a database. This data can be used in the future to automatically checking-in users to a particular location without them specifying one explicitly. Like Hunt The Fox, communication between a server and a client is done through asynchronous JSON messaging using a polling mechanism. The implementation and data model are based on the one from Redpin for both the client and server sides. The server side is im-

65 MEEX plemented in Java and thus can run on any operating system. The database on server side is SQLite, but any other database having Java Database Connectivity (JDBC) API can be used. Meex client and server can be found on the disc attached to this work. iphone Client The iphone client user interface was designed, as in the case of Hunt The Fox, to be as userfriendly as possible. It has three tab views: profile, check-in, and home. Users specify their name in the profile tab as well as the scope of chat, i.e. whether to get updates from other users in the same room, floor or building (Figure 3.6). The check-in tab is where users select their current location from a list of predefined locations (Figure 3.7 and Figure 3.8). The main tab is the home tab where users view posts from other users in their vicinity (Figure 3.9) and communicate with them by posting messages (Figure 3.10). The home tab view is updated every 60 seconds, so users do not have to explicitly refresh the messaging board to receive new posts. Figure 3.6: Profile tab. A user can fill in profile details and choose whether to get updates from users in the same room, floor or building. Figure 3.7: Check-in tab. Users select their current location. Figure 3.8: Check-in tab. Next level of hierarchy for selecting the location. Users are automatically checked-out upon exiting the application. Similarly to Hunt The Fox, Meex does not save user s device identifier or any other personal information.

66 CHAPTER 3. USER MOTIVATION STUDY 55 Figure 3.9: Home tab, message board view. Figure 3.10: Home tab, post a message view. 3.5 Conclusion Both Meex and Hunt The Fox applications are intended to motivate users to contribute their location and both do it in an implicit way, meaning users are engaged in the activity and are not necessarily aware of their contribution in training the localization system. This is in contrast, for example, to an application that changes mobile profiles base on the user s location within the office (discussed in Section 3.2). The user has to explicitly train such applications on various locations. In the latter case it is in the best interest of the user to provide correct location tags, while in the case of Meex and Hunt The Fox, users might cheat. Both applications seem to be similar, but they are quite different and were designed for different purposes. Hunt The Fox is designed for people who know each other and can play for fun or competition during their spare time, while the main idea of Meex is to mingle between people who do not know each other, such as people waiting for a delayed flight in an airport. However, Meex can still be used by people who know each other, for example two colleagues sitting at the same building on different floors and want to chat during work time. Hunt The Fox is played by relatively static groups of people, and Meex can be used by the same people in various situations and contexts, thus making the groups formed more dynamic. In addition, as was already mentioned, Hunt The Fox is most likely played for short time periods, and Meex could have a more continuous engagement over time. Meex by nature is of little use to one person, while Hunt The Fox is still enjoyable for a single player. From the design and implementation point of view, the applications are different as well. Hunt The Fox utilizes a graphical map to show physical locations, while Meex does not have one. In addition, Hunt The Fox performs instant labeling, and Meex performs interval labeling. We believe that both applications could draw interest of users and help in collecting fingerprints for crowdsourced-based applications.

67 CONCLUSION

68 4 Discussion 4.1 Conclusion In this work we presented an extensive study and analysis of the characteristics of WiFi signals collected by users over a long time period. Our study shows how the behavior of WiFi signals changes in indoor dynamic environment depending on various unpredictable factors. We illustrate how arbitrary the data could be for crowdsourcing-based systems. RSSI distribution might substantially change over different periods, duration, and frequency, invalidating the assumption that data distribution does not change much over time. We also showed that filtering techniques based on AP visibility might not work due to changes in APs visibility between measurement periods of different lengths. In addition, presence of people and their continuous movement affect WiFi behavior in unpredictable ways, which makes indoor environments even more unstable. At the end, we did not find any strict pattern to WiFi behavior, which led us to abandon the route of data-specific improvements. Following the WiFi signal study we evaluated three localization algorithms: Redpin, Bayesbased assuming a normal distribution, and SVM-based in terms of accuracy using the same data set collected by users. Our results show that the localization algorithm based on SVM is more accurate then others, while Bayes-based algorithm gave the less accurate results. Comparison of SVM and Redpin in terms of performance showed that Redpin s performance suffers for data sets of 2000 fingerprints and more. We introduced optimization technique for SVM where a prediction model is periodically rebuilt in the background and location calculation using the available model takes little time. Therefore, using SVM we improved both accuracy and performance of the localization system. In addition, our results show that interval labeling increases the accuracy of the algorithms. We implemented the optimized SVM-based localization algorithm for the Redpin System locator on the server side, and interval labeling on the existing Redpin iphone client application. We continued with collecting and classifying various ideas and incentives to motivate users in contributing their location. Two different applications were chosen and implemented: the game Hunt The Fox and the social network Meex. We conducted a user study to investigate 57

69 FUTURE WORK whether Hunt The Fox motivates users to contribute. The study spanned over 4 weeks, and two different groups of people took part in it. We discovered that indeed users were interested in the game and motivated to play, especially during the first week, however at the end we learned that most likely Hunt The Fox will not keep users playing in the long run. We also found that motivation was different among the groups. 4.2 Future Work As already mentioned, WiFi signals are significantly unstable as they depend on many unpredictable factors. However, to fully understand WiFi signal behavior in dynamic indoor environment, a much larger-scale research should be carried out including in different seasons of a year, measurements from various types of buildings, locations worldwide, a large set of the most widely spread mobile device models. User motivation to contribute is essential to crowdsourced-based systems. Incentives should be studied in depth and different applications should be developed for that purpose and compared. Our Hunt The Fox user study included only 9 users from two homogenous groups in two specific environments. It would be important to check other groups of users. In addition, due to time limitations, we run the study for 4 weeks, but it is not enough to fully understand whether an application motivates users to contribute over a long time period. It would be also interesting to evaluate users motivation over time on a longer-term application, such as Meex. A study of this kind requires large audience of users who are not familiar with each other and a long time period. We expect that users motivation will increase as more and more people are using the application. In this work we do not consider users contributions with wrong location tags. Users may contribute incorrect fingerprints either consciously or unconsciously. One of the open questions is what to do with ambiguous labels, where the same location is labeled differently by different users. These are important issues to overcome when taking the crowdsourcing approach. 4.3 Challenges It was interesting to work on this research and I am happy with my choice of subject. Especially since it consists of multiple parts; from pure research, through working with and analyzing large sets of data, to the development of iphone applications and user study. First of all, I learned a lot about indoor localization which is a wide, challenging topic with much to research and understand. It was fascinating to study the behavior of WiFi signals and to realize just how unpredictable they can be. I found it especially intriguing to study the many factors which effect WiFi signals, and how dynamic and constantly changing they are. For me it was challenging to work with a large collection of fingerprints, analyzing it, picking measurement intervals of various lengths. Especially since the data was collected in arbitrary, free form ways, as opposed to following a strictly structured plan. It was also a great experience to learn a new programming language, Objective-C, and developing applications for the iphone and Mac OS X. Working with people and studying their motivation through playing Hunt The Fox was also challenging. After they installed the game, we had no influence on whether or not they would

70 CHAPTER 4. DISCUSSION 59 actually play it, and how often.

71 CHALLENGES

72 A Accuracy of the Algorithms for Increasing Datasets A.1 Instant Labeling Number of measurements per location Bayes Redpin SVM

73 62 A.2. INTERVAL LABELING A.2 Interval Labeling Interval length in minutes per location Bayes Redpin SVM

74 B Instant vs. Interval Labeling For the instant labeling one sample corresponds to one measurement, for the interval labeling one sample corresponds to one interval. Number of 5 min 10 min 15 min 20 min 25 min 30 min instant samples Average improvement E-05 from the previous interval 63

75 64

76 C Hunt The Fox Survey 1. How often do you play HTF? Response Options Absolute Qty Relative Qty (%) Less than once a week % Once a week % Twice a week % Three times a week % More than three times a week 0 0% NO RESPONSE 0 0% Total non-empty responses 7 2. How do you play HTF? Response Options Absolute Qty Relative Qty (%) Chasing the fox % Waiting for the fox to arrive at my place % NO RESPONSE 0 0% Total non-empty responses 7 65

77 66 3. In days you do not play HTF, what is the main reason? Response Options Absolute Qty Relative Qty (%) Do not like it 0 0% Do not have the time % Forget about it % Out of office % Other, please specify: % NO RESPONSE 0 0% Total non-empty responses 7 Other, please specify: 1. Running another application. 2. Missing benefit. 3. Es ist umständlich. 4. Were you tempted to cheat (i.e., catch the fox although you were not in the same room)? Response Options Absolute Qty Relative Qty (%) Most of the time 0 0% Sometimes 0 0% No % Almost never 0 0% Never % NO RESPONSE % Total non-empty responses 6 5. It there would be a challenge every month, do you think you would play Hunt the Fox in the long run (i.e. would you still play after 6 or so month)? Response Options Absolute Qty Relative Qty (%) Yes 0 0% Probably 3 50% Don t know % Probably not % No 0 0% NO RESPONSE % Total non-empty responses 6

78 APPENDIX C. HUNT THE FOX SURVEY Were you more motivated after receiving the scores via ? Response Options Absolute Qty Relative Qty (%) Yes 7 100% No 0 0% I did not receive any s 0 0% NO RESPONSE 0 0% Total non-empty responses 7 7. Were you more motivated after learning there is a prize for the winner? Response Options Absolute Qty Relative Qty (%) Yes % No % I did not know there is a prize for the winner. 0 0% NO RESPONSE 0 0% Total non-empty responses 7

79 68 8. What is that you would most like to change about Hunt the Fox, or add new functionality, if any? User response 1. Getting current scores via mail or push notification (daily?) 2. Live high score alerts pushed to the device would motivate competition 3. Put more foxes at once, say pick three rooms and put a fox in each, so that if I m closer to a room I pick that, or don t need to go in my professor s office while he s having a meeting to... catch the next fox! :D. It could also work that you see 5 foxes, and need to catch at least 3 or 4 until you get another batch of 5. Also, it would help if you could see the scores from the other players real time also, instead than in one once a week. The prize incentive really helped. 4. Fox should not appear in rooms which one cannot access (i.e., service rooms). Although that kind of contradicts the essence of the game for you (?), it would be nice if the App would automatically catch the fox without the need for pressing the catch button. 5. There was a bug, when I click catch button at the right place, it said I am cheating. It happened at Axel s seat and at the sitting table in the middle of the office. 6. Seems like the fox appears at quite a few predefined locations in the rooms. Would be much more fun if there were more locations, maybe 25 or so at the Office. Would be even more fun if it moved constantly (animated). 7. It should be possible to displace the fox e.g. two people standing behind the fox pushing the displace button and the fox will run in the other direction... and of course... the 4th hunter is waiting there to catch it. :-) so this game will be more group orientated... Total non-empty responses 7

80 D Affidavit I hereby declare that this master thesis has been written only by the undersigned and without any assistance from third parties. Furthermore, I confirm that no sources have been used in the preparation of this thesis other than those indicated in the thesis itself. This thesis has not yet been presented to any examination authority, neither in this form nor in a modified version. Zürich, 30th April 2010 (Place), (Date) (Signature) 69

81 70

82 E Statement regarding plagiarism when submitting written work at ETH Zurich By signing this statement, I affirm that I have read the information notice on plagiarism, independently produced this paper, and adhered to the general practice of source citation in this subject-area. Information notice on plagiarism: s en.pdf Zürich, 30th April 2010 (Place), (Date) (Signature) 71