SPRINGER BRIEFS IN COMPUTER SCIENCE. Ruipeng Gao Fan Ye Guojie Luo Jason Cong. Smartphone- Based Indoor Map Construction Principles and Applications

Transcription

1 SPRINGER BRIEFS IN COMPUTER SCIENCE Ruipeng Gao Fan Ye Guojie Luo Jason Cong Smartphone- Based Indoor Map Construction Principles and Applications

2 SpringerBriefs in Computer Science Series editors Stan Zdonik, Brown University, Providence, RI, USA Shashi Shekhar, University of Minnesota, Minneapolis, MN, USA Xindong Wu, University of Vermont, Burlington, VT, USA Lakhmi C. Jain, University of South Australia, Adelaide, SA, Australia David Padua, University of Illinois Urbana-Champaign, Urbana, IL, USA Xuemin Sherman Shen, University of Waterloo, Waterloo, ON, Canada Borko Furht, Florida Atlantic University, Boca Raton, FL, USA V. S. Subrahmanian, University of Maryland, College Park, MD, USA Martial Hebert, Carnegie Mellon University, Pittsburgh, PA, USA Katsushi Ikeuchi, University of Tokyo, Tokyo, Japan Bruno Siciliano, University of Naples Federico II, Napoli, Italy Sushil Jajodia, George Mason University, Fairfax, VA, USA Newton Lee, Woodbury University, Burbank, CA, USA

3 SpringerBriefs present concise summaries of cutting-edge research and practical applications across a wide spectrum of fields. Featuring compact volumes of 50 to 125 pages, the series covers a range of content from professional to academic. Typical topics might include: A timely report of state-of-the art analytical techniques A bridge between new research results, as published in journal articles, and a contextual literature review A snapshot of a hot or emerging topic An in-depth case study or clinical example A presentation of core concepts that students must understand in order to make independent contributions Briefs allow authors to present their ideas and readers to absorb them with minimal time investment. Briefs will be published as part of Springer s ebook collection, with millions of users worldwide. In addition, Briefs will be available for individual print and electronic purchase. Briefs are characterized by fast, global electronic dissemination, standard publishing contracts, easy-to-use manuscript preparation and formatting guidelines, and expedited production schedules. We aim for publication 8 12 weeks after acceptance. Both solicited and unsolicited manuscripts are considered for publication in this series. More information about this series at

4 Ruipeng Gao Fan Ye Guojie Luo Jason Cong Smartphone-Based Indoor Map Construction Principles and Applications 123

5 Ruipeng Gao Beijing Jiaotong University Beijing China Fan Ye Stony Brook University Stony Brook, NY USA Guojie Luo Peking University Beijing China Jason Cong UCLA Los Angeles, CA USA ISSN ISSN (electronic) SpringerBriefs in Computer Science ISBN ISBN (ebook) Library of Congress Control Number: The Author(s) 2018 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Printed on acid-free paper This Springer imprint is published by the registered company Springer Nature Singapore Pte Ltd. part of Springer Nature The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore , Singapore

6 Preface This book focuses on the ubiquitous indoor localization services by specifically addressing the issue of floor plans. It combines the computer vision algorithms and mobile techniques to reconstruct the complete and accurate floor plans, thus providing better location-based services for both humans and vehicles via commodity smartphones in indoor environments (e.g., a multilayer shopping mall with underground parking structures). This book is based on Dr. Ruipeng Gao's PhD thesis co-supervised by Professors Jason Cong, Guojie Luo, and Fan Ye. The authors would like to thank the supports from NSFC and PKU/UCLA Joint Research Institute of Science and Engineering. Beijing, China January 2018 Ruipeng Gao v

7 Contents 1 Introduction of Indoor Map Construction Introduction... 1 Reference Indoor Map Construction via Mobile Crowdsensing Introduction Design Overview Landmark Modeling The Landmark Model Coordinates of Geometric Vertices Connecting Points of Wall Segments Example Landmark Placement Notations Spatial Relation Acquisition Problem Formulation Optimization Algorithm Map Augmentation Wall Reconstruction Hallway Reconstruction Room Reconstruction Connection Area Detection Types of Connection Areas Features Unsupervised Classification Refinement and Placement Types of Connection Areas Performance vii

8 viii Contents 2.8 Discussion Related Work Conclusion References Incremental Indoor Map Construction with a Single User Introduction Overview Localization via a Single Image Trajectory Calibration and Cleaning Trajectory Calibration Trajectory Cleaning Map Fusion Framework Dynamic Bayesian Network Particle Filter Algorithm Landmark Recognition Compartment Estimation Performance Discussion Related Work Conclusion References Indoor Localization by Photo-Taking of the Environment Introduction Relative Position Measurement Triangulation Method User Operations and Location Computation Criteria for Users to Choose Reference Objects Robustness of the Localization Primitive Site Survey for Reference Objects Coordinates Location Estimation in Unmapped Environments Experiments on Site Survey Identifying Chosen Reference Objects System Architecture and Workflow Benchmark Selection of Reference Objects Benchmark Selection Problem NP-Completeness Proof A Heuristic Algorithm Improve Localization with Geographical Constraints Experiment Results and Problems in Early Prototype Geographical Constraints System Localization Performance Discussion... 76

9 Contents ix 4.9 Related Work Conclusion References Smartphone-Based Real-Time Vehicle Tracking in Indoor Parking Structures Introduction Design Overview Trajectory Tracing Conventional Approaches Shadow Trajectory Tracing Equivalence Proof Real-Time Tracking Intuition Road Skeleton Model Probabilistic Tracking Framework Tracking Algorithms Landmark Detection Types of Landmarks Feature and Classification Algorithm Prediction and Rollback Performance Discussion Related Work Conclusions References

10 Chapter 1 Introduction of Indoor Map Construction Abstract We describe the motivation and background of map construction for ubiquitous indoor location-based services, and then give an overview of this book and present how it is organized in the following chapters. 1.1 Introduction In contrast to the almost ubiquitous coverage outdoors, localization service is at best sporadic indoors. The industry state-of-the-art Google Indoor Maps [1] as of 2017, covers 10,000 locations worldwide, which is only a small fraction of millions of indoor environments (e.g., airports, train stations, shopping malls, museums, and hospitals) on the Earth. The lack of indoor maps is a fundamental obstacle to ubiquitous indoor location-based services. Service providers have to conduct effort-intensive and time-consuming business negotiations with building owners or operators to collect the floor plans, or wait for them to voluntarily upload such data. Neither is conducive to large-scale coverage in short time. Recently, some academic work has made admirable progress to automatic indoor map construction. They require only commodity mobile devices (e.g., smartphones); thus, can achieve scalable construction by crowdsensing data from many users. Despite such progress, they usually require inertial and WiFi data, which are inherently noisy thus difficult to produce precise and detailed maps. In this book, we leverage both mobile and vision data via commodity smartphones to construct the floor plans of complex indoor environments. It avoids the intensive efforts and time overhead in the business negotiation process for service providers. They do not need to talk to building owners/operators one by one, or hire dedicated personnel to measure indoor environments inch by inch. It opens up the possibility of fast and scalable floor plan reconstruction. After a comprehensive review of scene reconstruction methods, this book produces accurate geometric information of each landmark from images and derives landmarks spatial relationship and the rough sketches of accessible areas with inertial and WiFi data for less computing overhead. Then, recent novel findings from the authors are presented in detail, including the optimization and probabilistic formulations for more solid foundations and better robustness to combat errors, sev- The Author(s) 2018 R. Gao et al., Smartphone-Based Indoor Map Construction, SpringerBriefs in Computer Science, 1

11 2 1 Introduction of Indoor Map Construction eral new approaches to promote the current sporadic availability of indoor locationbased services, and a holistic solution from floor plan reconstruction to indoor localization, tracking, and navigation. Our new approaches are designed for different types of indoor environments (e.g., shopping malls, office buildings and labs) and different users. Our design is based on the realization that computer vision and mobile techniques have complementary strengths. The vision techniques can produce accurate geometric information when the area has stable and distinct visual features. They are suitable for landmarks where logos, decorations constitute rich features, and detailed information about their positions/orientations are desired. The mobile techniques give only rough sketches of accessible areas with much lower computing overhead, which are suitable for in-between sections such as textureless or glass walls where much fewer stable features exist, while less detailed information is required. Therefore, we leverage expensive vision techniques to obtain more accurate and detailed information about individual landmarks, and use cheap inertial data to obtain the placement of landmarks on a large, common floor plane, and derive the less critical hallway and room information at lower fidelity. The optimization and probabilistic formulations give us more solid foundations and better robustness to combat errors from data. The reconstructed indoor maps can significantly improve the indoor locationbased services, both for pedestrians and drivers in indoor environments. With the reconstructed maps, we explore alternative indoor localization approaches which have comparable performance but without relying on the RF signature. Specifically, we leverage environmental landmarks (indoor places of interests, such as logos of stores, paintings on the walls, or bumps in the parking structure), and users use the smartphone to measure their relative positions to those landmarks, and the coordinates of the reference landmarks are used to compute user locations. This has a few advantages: (1) Landmarks are part of the environment and abundant; they do not require dedicated deployment and maintenance efforts like IT infrastructure; (2) they seldom move and usually remain static over long periods of time. They are not affected by and thus impervious to electromagnetic disturbances from microwaves, cordless phones, or wireless cameras. Once measured, their coordinates usually do not change, thus eliminating the need for periodic recalibration. The rest of this book is organized as follows: First, we present two methods to construct complete indoor floor plans automatically, and then give two examples on how the produced maps improve indoor location-based services. Specifically, Chap. 2 leverages the crowdsensed data from mobile users and constructs the floor plan as a batch processing system, and Chap. 3 generates it incrementally from a single user. With the reconstructed maps, Chap. 4 localizes pedestrians in the building without RF signals, and Chap. 5 tracks vehicles in real time in parking structures. Reference 1. Google Indoor Maps Availability, answer=

12 Chapter 2 Indoor Map Construction via Mobile Crowdsensing Abstract The lack of indoor maps is a critical reason behind the current sporadic availability of indoor localization service. Service providers have to go through effortintensive and time-consuming business negotiations with building operators, or hire dedicated personnel to gather such data. In this chapter, we propose Jigsaw, a floor plan reconstruction system that leverages crowdsensed data from mobile users. It extracts the position, size, and orientation information of individual landmark objects from images taken by users. It also obtains the spatial relation between adjacent landmark objects from inertial sensor data, and then computes the coordinates and orientations of these objects on an initial floor plan. By combining user mobility traces and locations where images are taken, it produces complete floor plans with hallway connectivity, room sizes, and shapes. It also identifies different types of connection areas (e.g., escalators, stairs) between stories, and employs a refinement algorithm to correct detection errors. Our experiments on three stories of two large shopping malls show that the 90-percentile errors of positions and orientations of landmark objects are about 1 2 m and 5 9, while the hallway connectivity and connection areas between stories are 100% correct. 2.1 Introduction Online digital maps (e.g., Google Maps) have provided great convenience for location-based services (LBS) outdoors such as finding nearby point-of-interests (POIs) and navigation. However, for indoor environments where people spend over 80% of the time [1], such maps are extremely scarce and unavailable in most buildings. This has become a huge obstacle to pervasive indoor LBS. Accurate indoor floor plan construction at low costs is urgently needed. Autonomous robots equipped with high-precision special sensors (e.g., laser rangers [2], depth cameras [3], sonars [4]) can produce high-quality maps. However, the high manufacturing costs, operational, and logistic obstacles make it difficult to deploy robots in large quantities. In this chapter, we propose Jigsaw [5], which leverages crowdsensed data from mobile users to construct the floor plans of complex indoor environments. It avoids The Author(s) 2018 R. Gao et al., Smartphone-Based Indoor Map Construction, SpringerBriefs in Computer Science, 3