Indoor Localization with Smartphones

Transcription

1 Indoor Localization with Smartphones Mid-Project Report Fall Semester Full report - Members: Aubrey Davis Tyler Fowler Ben Gratias Connor Wagener Graduate Student Overseer: Saideep Tiku Department of Electrical and Computer Engineering Colorado State University Fort Collins, Colorado Project advisor: Approved by: Dr. Sudeep Pasricha Saideep Tiku

2 Abstract: For years, GPS has been used by civilians and companies alike as an accurate positional service. This is used for purposes such as finding directions, targeting missiles, or controlling robots. However, GPS has several large limitations, such as having a margin of error of several meters and having trouble penetrating into buildings. Indoor localization refers to techniques of solving this deficiency of GPS and determining precise indoor position. Indoor localization is a relatively-new, imperfect technology that is not yet viable enough to break into the market at large. Indoor localization can be useful for giving directions to navigate buildings, or even for use by emergency services to help find trapped civilians, which is why many companies are actively researching the technology. However, taking measurements can be expensive, time consuming and inaccurate. Currently, this is done by humans over multiple trials in many different locations. Human inaccuracies can lead to sensor drift that will result in the accumulation of large errors over short distances, and large time commitments of this process will lead to high labor expenses. In order to avoid these pitfalls, we have created a mobile computing solution to perform calculations and accurate data collection. A semi-autonomous robot built into a Roomba Create frame can collect unique wifi samples, also referred to as fingerprints, and compile information on a building as part of a complete localization solution. This project uses smartphone sensors to take measurements of wi-fi signal strength and direction. This information will later be interpreted by machine learning algorithms in order to extrapolate current position on user devices. Major issues in fingerprinting for the use of indoor localization include the cost and length of time needed for the data collection. Our robot will allow for low man hour, low cost, highly accurate fingerprinting that can be used by machine learning algorithms to extrapolate precise position. This robot would make the setup procedure fast and accurate and greatly reduce cost of indoor localization, the two barriers we believe are most important to overcome to allow for widespread use of this technology. Our semi-autonomous robot is capable of following a line adorned with equally spaced markers using computer vision through OpenCV on a Raspberry Pi microcontroller. The robot is able to detect these markers with color detection capabilities. The robustness of the line following and detection allows for long shifts of fingerprinting, upon which the Roomba will utilize its inherent functionality to search out its charging dock and repeat the process. The team has created a platform to work on, and progress should increase on the second In the coming semester, autonomy of the drone will increase. We will make use of additional computer vision software to utilize markers and waypoints to allow the drone to keep note of checkpoints and perform more accurate fingerprinting faster. Additionally, the phone app created to gather signals must be integrated onto the Roomba frame, and information must be stored on the Raspberry Pi for later use. 1

3 Table of Contents: Abstract.. Page 1 List of Figures and Tables.. Page 3 1. Introduction...Page Objectives.... Page Primary..Page Secondary.. Page 5 2. Summary of Previous Work.. Page Design...Page 6 3. Hardware and Software.. Page Hardware..Page Software... Page Sampling Code..Page Motion Code.... Page Sensing Code.... Page Conclusions and Future Work... Page Conclusions..Page Future Work......Page 13 References Page 15 Appendices: Appendix A (Abbreviations)...Page 15 Appendix B (Budget)...Page 15 Appendix C (Timelines and Deliverable Documents)...Page 17 Appendix D (irobot Create 2 Op Codes)...Page 18 Appendix E (Sample Codes)...Page 19 Acknowledgements Page 21 2

4 List of Figures and Tables: Number Title 1 Table 1: Table of Tables and Figures 2 Table 2: Projected Budget 3 Table 3: Actual Budget 4 Table 4: Project Timeline 5 Table 5: Create 2 Op Codes 6 Figure 1: Modified Create 2 7 Figure 2: Robot with Tripod 8 Figure 3: Underside view of the robot, showing mounted Pi camera 9 Figure 4: Robot with modified bin and components (Pi and power bank) 10 Figure 5: Standalone 3-D printed bin with components, clearer view of Raspberry Pi 11 Figure 6: Tripod added to the top, note how it is placed 12 Figure 7: Line Detection Code Sample Image 13 Figure 8: Color Detection Code Sample Image (Table 1: Table of Tables and Figures) Chapter 1 - Introduction: A major breakthrough during the 20th century was the creation of an effective global navigation system by the US military. This would later be known as GPS, and allowed for navigation nearly everywhere on the globe. However, due to the passive nature of the satellites used for triangulation, signal strength cannot be picked up reliably inside of buildings. Another limitation of GPS technology indoors is the low level of accuracy achievable. GPS technology outdoors has an average accuracy of only 8 meters. This is not an acceptable level for most indoor applications where an error of only one meter can place the user in an incorrect room. As a result, research began into indoor localization, a system allowing for precise indoor navigation abilities. 3

5 An important consideration when using a phone to perform indoor localization is in regard to limited hardware resources. For instance, reliance on battery forces a low clock rate, slower RAM, and similarly scaled peripherals to save power. Additionally, a program run on a phone should maintain minimum load and maximum efficiency in order to conserve battery life while remaining responsive and accurate. Phones do not have space for a fan, and must not generate too much heat. This forces phone manufacturers to scale processing power as well. Localization accuracy is an important factor when dealing with small corridors and rooms in buildings. Differences of only a foot or a few inches could place a user in an incorrect room or give them faulty directions. Errors are inherent in all methods of indoor localization, therefore this project seeks to find an approach where localization errors can be minimized. For example, the dead reckoning technique uses motion sensors in phones to extrapolate movement of a user. This can lower system requirements, but results in sensor drift and error accumulation over time that leads to large error margins. Our method, fingerprinting, can be very accurate assuming constant conditions. Though due to variations in Wi-Fi signal during different times of day due to use, foot traffic, etc., similar errors can be accumulated or misinterpreted by machine learning algorithms applied to data. In order to minimize error, several tactics can be used. Wider breadth sensing records more signals and can improve the uniqueness of each fingerprint, keeping navigation apps from confusing coordinates with one another. Multiple rounds of fingerprinting can eliminate outliers and allow for determination of differences in signal strength during different times of day. This way, only large differences in signal strength such as outages are able to create widespread error. This project proposes a similar strategy of fingerprinting to set up a building for indoor localization. Several issues arise when performing complete fingerprinting of a building. Several devices must be carted around by a multi-member staff, stopping at many exact points to take and record measurements. This process takes hours and is extremely tedious, even in small buildings. In a large scale rollout of this solution, hiring a skilled crew for several shifts would be expensive and unrealistic. Additionally, any changes to the building or Wi-Fi network would require a complete repeat of this process and the expenses inherent therein. In order to solve this problem, we have started work on a semi autonomous robot which will minimize data collection time, cost, and human error, and maximize repeatability and ease of use. Using a robot instead of a staff force would critically provide very consistent measurements. Constant height, distance between measurements, and lack of motion during measurement would be easier to obtain. By removing the human aspect of data collection as much as possible, wifi samples in a broader scale are easier to obtain without as much error. Reliable measurements are crucial for training machine learning algorithms, and even small irregularities could lead to overcorrection and error in positions displayed to a user. Thus the primary objective of robot use for data gathering is the elimination or minimization of extra environmental variables. In order to avoid sensor drift and maintain a very small margin of error along an entire route, we have used computer vision to create very effective path tracing software capable of following an exact path as required within a minimal margin of error of only a few centimeters. Our work, exemplified below, serves as both a prototype and a proof of concept, and will quickly grow and improve as an indoor localization solution over the coming months now that a solid platform has been developed. Additionally, the use of a Roomba Create 2 frame, which will be 4

6 explained in greater detail below, allows for faster prototyping and a variety of sensors to utilize in the future. We expect this robot to be critical to the growth and wide use of indoor localization. Companies are well aware of the possible benefits of mapping their stores and offices for minimization of risk during disasters as well as increasing customer service and the efficiency of sales floor staff, but are hampered mostly by price. The majority of businesses interested in an indoor localization solution are low margin bulk sales storefronts such as department stores and supermarkets. The low cost of our design and the lack of staff members needed to perform fingerprinting would make indoor localization a very tempting prospect, and one that could be completed quickly with minimal customer interruption. If this project were to be put into production, it is clear that costs would remain low. Assuming no licensing issues with the Roomba platform, it is likely that the finished product will cost considerably less than $1,000 dollars for materials. Keeping the modular design setup as we have attempted to design allows for manufacturing to focus almost entirely on the bin area of the Roomba, as all components are contained within that section. This would minimize costs of labor as well. Additionally, this business plan of using such a product would likely involve a leasing strategy rather than a permanent sale, as many companies would not have much use for the product after initial fingerprinting. A finished design with all current and future features as described in this report would be a very tempting prospect to most large companies for use in storefronts, offices, or otherwise. Over the course of the rest of this paper, primary objectives of the project will be discussed, as well as a summary of previous work and more details on design decisions. The hardware utilized and its communication, specs and usage will also be detailed. Software will be explained and results of current programming displayed. Specific portions of code will be laid out in detail and the overall software structure will be displayed and broken down. This will be followed up with a general conclusion and analysis as well as clarification on future work Objectives Primary : We aimed to create a robot that allowed for both faster and more precise fingerprinting of a building. This device was created to be capable of producing repeatable results to be able to create a much more precise map of the target building through multiple samples taken at different times. The final result must also be capable of accomplishing this task with less human effort and time investment than manual fingerprinting requires, and provide adequate information to train machine learning algorithms Secondary : In order to increase the usefulness of the design, a satisfactory balance between power consumption and total energy stored in battery banks must be obtained. Initial design seeked to achieve a minimum of two hours of uptime. This goal was achieved mainly by keeping track of what power consumption each component was using and making sure we tested with the battery fully charged. If later renditions of the project cause battery life issues, then we will consider ordering a new secondary backup battery of the LiPo variety. The Roomba battery life has not proved itself to be an issue so far, and has been run for longer periods with all vacuum 5

7 components removed. Another secondary goal of the project is to maintain large portability of the end design which would allow for further increased efficiency and reuse of the robot. We would like for the robot and app to be applied to a variety of phones, so that it can apply more closely to whatever technology a user has on hand. Chapter 2 - Summary of Previous Work: This project is still in its first semester, as it began in the fall of Due to this there was no work previously done on any of the process or designs that we have worked on this semester Design: Our design utilizes an irobot Create 2 Programmable Robot, which is a development version of a traditional Roomba vacuum. This will allow us a superior level of flexibility that the creation of our own chassis would not. For instance, this strategy allows for tremendous cost saving measures at only $200, while saving large amounts of development time. By using a Roomba, we will have access to a large array of reference materials and guides on the internet. This sped up our work for the controls and helped to avoid roadblocks. Additionally, this product is backed by warranty and is pre-tested and ready for out of the box use. If we were to build our own chassis, cost estimates had put costs for similar sensors, motors, and control at over $200, making this solution a net gain in all respects. The irobot Create contains a variety of sensors that we are able to access and get feedback from, knowing things such as when the bumper is pressed or the internal temperature is too high. It also can measure the speed of the robot. The stepper motors within the robot allow for precise turn radiuses per motor, which is vital to making accurate corrections when needing to follow a specific path through a building. Our plans for modifications to the irobot Create 2 to make it viable for our project included changes to both the hardware and software of the robot. We removed parts of the vacuum within the bin below the robot and 3-D printed a new bin to work as more of a drawer to hold other components and make good use of this space. The bin could then hold both the microcontrollers and wiring. Other hardware additions included mounting a camera to the bottom of the robot for video processing and input, as well as a way to mount the phone(s) with the data collection app upon the robot so that they can be raised up to enough distance to nearly mimic a human holding a phone in hand, but also to keep the height uniform. We were able to use a camera tripod for the first prototype, which works as an adjustable platform for a single phone that weighs enough to not need to be permanently secured to the top. For the software modifications, we chose the Raspberry Pi 3 for our processor and wrote custom coding in Python, based on OpenCV resources. The robot communicates with the processors using a serial connection, so that we can control the robot s parts and movement. Figure 1 below shows the robot with its modified bin and the Raspberry Pi inside. The modified icreate 2, in its completed state for our first prototype, contains the Pi and the battery block to power the Pi within the bin, which is connected by serial cable to the robot. The bin is secured by clicking it into place to avoid slipping while moving. The camera tripod is extended and fixed to a general height of around a meter, then placed on top of the robot with its 6

8 legs propped against the edges of the top cover. It is a heavy enough part that it sits without needing to be permanently secured, so we can take it off when not running it for testing. The phone can then be mounted on the top of the tripod. This setup of the framework for the testing bot can be seen in Figure 2 below. Figure 1: Modified Create 2 Figure 2: Robot with Tripod Chapter 3 - Hardware and Software: Hardware: The Roomba has an array of sensors that allows for good positioning and strong sensor fusion. The robot has a pressure sensor in the front capable of detecting objects and walls and minimizing collision force. On the same plate is a set of low distance infrared sensors that allow the Roomba to detect objects from a short distance. Its wheels are powered by stepper motors, which will allow for accurate distance calculations, especially when used in conjunction with color sensing on the dual colored front wheel. Additionally, the development version does not include vacuum components. This will allow extra space for sensors and boards and protect them from damage. The lack of power draw to such components will lengthen battery life. Without a running vacuum, vibration and noise 7

9 will also be minimized and accuracy of measurements can be improved. More structural connections are provided by the development Roomba than in a regular version as well as included plans for 3-D printed mounts. A guide is provided illustrating safe drill points where connections can be made without fear of hitting a PCB or wire. This offers us a lot of flexibility when designing mounts. The Roomba will allow us to lower computational power needed elsewhere in the design. Originally we had planned to use two Raspberry Pi boards to control this robot, one for driving and sensing and the other for interfacing with smartphones used to take measurements. We were able to eliminate one of these boards used for driving and combining it with the sensing and app connection, allowing for simpler wires, less power use, and an overall streamlined design. Design will begin using this rugged, wide chassis. The Roomba s stepper motors help keep track of distance. We will also use this Raspberry Pi for the bulk of storage and computation. With a quad core processor running at 1.2 Ghz, we expect this to be more than necessary. The Pi also has 1GB of RAM for storage, which makes it viable as a storage for the data samples of Wi-Fi fingerprints the app collects. The Raspberry Pi will need to interface with the sensors and the phones used for fingerprinting, make sense of all data, and direct the movement of the drone accordingly. We chose a video camera that is made to work with the Pi for viewing lines and markers on the ground. In the initial version of our project, we had the camera working to follow pre-set ground markings, such as a brightly colored tape line or rope. Later, we will add more functionality and sensing in order to eliminate many of the markers and create a more autonomous drone. The camera was secured on the bottom side of the robot, then later we covered up the infared lights that it came with to get less flooding of light and more color feedback. This was temporarily done with pieces of paper taped over the lights on either side of the camera. Cell phones are mounted a few feet above the ground to simulate the height a person would hold a phone at. This is why a wide and dense chassis is necessary, as the drone could become quite topheavy. In our first prototype, we chose to implement a camera tripod. Phone mounts on top created with 3-D printing will eventually allow for the phones to pivot in all directions and fingerprint data in multiple orientations. For now it can be settled in one set orientation on the top. The robot s battery works long enough by itself, so we have not invested in the LiPo battery that was originally planned to backup the robot s power. The irobot Create 2 is also compatible with all addons for the Roomba, including a larger battery, so that s also an option to upgrade in the future. The Raspberry Pi works off of a Kmashi power bank that many people use to recharge their phones, which is also how we will power the phones if they need to charge while mounted for long data collection processes. The camera is powered through its connection to the Pi. The drone will move through the building and stop whenever necessary to take measurements. Fingerprints from each phone are stored locally, and exported to the Raspberry Pi upon completion of measurement. Following this section, we have included some clearer pictures of our robot and the hardware components. 8

10 Figure 3: (above) Underside view of the robot, showing mounted Pi camera Figure 4: Robot with modified bin and components (Pi and power bank) 9

11 Figure 5: Standalone 3-D printed bin with components, clearer view of Raspberry Pi Figure 6: Tripod added to the top, note how it sits with legs propped against rim of bot 10

12 3.2 - Software: The Create 2 code used in this project can be divided into three main portions, each of which are responsible for a specific function pertaining to the localization process. The sampling code, responsible for the measurement, quantization, and storage of the wifi signals used in the fingerprinting process, and the autonomous motion code, which controls movement and which receives input from final portion, the exterior sensing code, which translates the world into something useable by the bot. Samples of all code are contained within the Appendix E Sampling Code: The ultimate purpose of this project is to be able to take accurate and repeatable samples of Wi-Fi signals. While the repeatability comes largely from the physical control of the bot, most of the accuracy is accomplished through sampling with actual phones instead of a more idealized sensor. This necessitates the development of an app to facilitate data collection on an Android device. The data collected on the periphery devices is then transferred onto the central Raspberry Pi in order to make the data easily accessed and utilized. The code responsible for completing these steps can be categorized into the following two primary sections: code running on the phones, code responsible for the data transfer between phone and Raspberry Pi and code running on the Pi to process data. The Android code was created in Android Studio. It requires permissions to access location and storage in order to take and store its readings. The app is triggered to take a reading on a button press. Once a reading is taken, the total output is stored in one file and each individual data type, i.e. SSID and signal strength, are also separated out for ease of access. The process of retrieving the captured data is accomplished by two more pieces of code. The first of which is a bash script that utilizes the Android Debug Bridge or ADB to repeatedly poll the connected android device for new copies of its output data. Separate code running on the Pi then analyzes the files returned by ADB for changes. If it detects that the returned data contains new information, it will process that data and update its internal database Motion Code: The movement code utilizes a serial input port on top of the Create 2 to feed it commands based on a predetermined instruction set built into the Create. There are dozens of different codes the robot can interpret and turn into a workable commands, a list of which can be found in Appendix D. It doesn t stop there though. You can also receive feedback from the machine and use that to influence the behavior of the bot. For example voltage and current measurements can be used to monitor battery consumption, allowing for multiple mid fingerprinting charges 11

13 extending the autonomous time period of the bot. Temperature can be monitored to keep hardware components within safe parameters and even be used to trigger cooling within the bot in the form of a remaining dust bin fan. This code works very closely to the sensing code, which is where bot input is fed and analysis of the working conditions are computed, as well as bot commands based on the input of the sensing code Sensing Code: The exterior sensing code is the heart of this project s functionality. Taking in input, as well as feeding commands, to both the motion and sampling protocols, the sensing code makes sure that everyone is doing what they should be when they should be. Currently, the main input to this system is a small camera mounted to the underside of the Create 2. This relays visual input that is then used for line detection, to keep the bot on track, as well as color detection, to allow for additional stimuli for specific commands. For example, currently we are focusing on searching for the color blue. When blue is spotted within a specific range of the camera s image, the bot is told to stop, and samples commence. When sampling has concluded and it is time to move on, a sampling complete input is received and the motion controls are sent back to work. In the future this code will be use for more accurate sensing, most likely in the form of sonar, physical input from the Create 2 bumpers, and a broader range of additional stimuli to allow for more precise commands to be delivered to the other systems based on the world around the bot. Figure 7: Line Detection Code Sample Image (left) Figure 8: Color Detection Code Sample Image (right) 12

14 Chapter 4 - Conclusions And Future Work: Conclusions: We have made a lot of progress over our first semester of work. The bot has gone from a concept to a physical reality. It has the groundwork for all of the advanced functions completed as well as much of the base functionality required for its main objectives. Currently the bot can move around a space on a predetermined path and take samples of wireless signal intensities, and will be further developed to create our end product. As for meeting the main objectives the results are as follows: A bot that can fingerprint faster and more accurately than a human has yet to be proven. Testing on accuracy of results and fingerprinting time has yet to be documented. Once integration of sampling software with sensing software is completed, this can be easily tested. The battery life is almost at exactly two hours, which satisfies another design goal. We will also take advantage of the Roomba s inherent ability to navigate back to its dock and recharge to essentially extend this battery life forever. Research is also being done into the purchase of larger battery packs. Obstacle avoidance is a success. Since the path is determined by a line or rope, as long as proper setup is followed there will not be any problems with collision or the potential for the robot to become stuck. Sampling is occurring and shows enough uniqueness to act well for fingerprinting purposes. From this point forward we will mostly work towards broadening the scope of the robot's functions, feedback and controls in an effort to make the bot more autonomous and thus be more helpful towards any group that wishes to perform fingerprinting. We will also be improving and refining data collected by the sampling software to obtain further accuracy and reduce outliers Future Work: Due to the inherent time restraints of the project, we were unable to complete everything that we wanted to implement. With this in mind we would like to create two more generations of the bot prototype that will allow for the increased control and functionality desired. In the first generation, path tracing protocol requires human setup and teardown of a location slated for fingerprinting. We also are not currently sampling everything possible, and samples are saved as 13

15 raw data and not properly analyzed before being stored on the Pi. The phone mounts are not fully completed, and will need to be finished before phones can be correctly installed. Generation two of the bot will still function under the same path tracing movement protocol that generation one worked on, but with increased functionality. More corrective steps will allow for more specific corrections to a given situation and allow for both slow and quick movements to correct errors such as drift. We will also be implementing a calibration feature that allows for the line detection software to work in a more varied set of conditions. This will include settings such as rope color, ground color, lighting conditions, etc, as well as a sensitivity settings for dealing with issues such as similarity in color between the rope and the floor. We will also be utilizing the query functionality of the bot to collect data on the status of the robot during the sampling procedure. This will include: Temperature readings to keep the Raspberry Pi properly cooled Voltage, current and remaining battery charge Bump sensor readings for impact detection Lift detection to reset the robot and allow it to continue to receive commands Gen two will also see the completion of the phone mounting system and integration of these phones with the Raspberry Pi. This will increase the pool of data from each sample significantly and allow for a better end product for the consumer. With the phone integration we will be utilizing the full capacity of our sampling software across multiple platforms to get a better understanding of the actual wireless signals within an area. Finally with Generation three we will be moving away from the path following software all together. This will see the implementation of a more autonomous control scheme to nearly eliminate setup and teardown time for data collection. We will be using sonar based sensors as well as physical detection to sense the direction and distance of structures within a space. This will help prevent errors such as bumping or becoming stuck on objects within a room. The robot can utilize a map of a location, a map scale, and the X and Y coordinates of the locations that need to be sampled to allow for movement without the use of a line. The sonar and bump sensors will also help prevent error accumulation as at any point in time the bot can simply check the distance to objects represented on the map to make sure it is on track. We would also like to be able to increase the range and type of frequencies that we are able to sample so that we can get a more complete picture of the environment that we are trying to fingerprint. Due to the tendencies of deep learning algorithms to produce better information as more data is put into the system, redundancy and a wider scope is beneficial. With more samples at varying frequencies, we can obtain great amounts of accuracy that would not be possible on just 2.4 GHz Wi-Fi alone. This will be further addressed in the Gen three version of the bot. 14

16 References: We have no references as of yet. Appendices: Appendix A (Abbreviations): Gen: Generation GPS: Global Positioning System LiPo: Lithium Polymer RAM: Random Access Memory Op: Operation PWM: Pulse Width Modulation App: Application SSID: Service Set Identifier ADB: Android Debug Bridge Appendix B (Budget): Our budget is set at $200 per team member, $800 in total. We will stay under this value as much as possible, but can see about getting a grant for assistance if we have parts that may push us over this budget. We plan to look into support from our advising professor, Dr. Pasricha, who may be able to find those willing to support our project if base funding is not sufficient. Unallocated funds will act as a cushion for potential overspending in any area of the project. Aubrey has been overseeing and tracking the budget as the project progresses. Currently most items are purchased through Dr. Pasricha. Budget outline is included on the next page. 15

17 Budget Allocation (Projected): Budget Item Budget Amount Chasis 25% Movement and Correction 35% Sampling and Storage 10% Prototyping 10% Unallocated 20% Table 2: Projected Budget Actual Budget Usage: Part Cost total Payment irobot Create 2 $ 200 Dr. Pasricha Raspberry Pi (x2) $ 70 Dr. Pasricha Charging Cable, micro USB $ 10 Dr. Pasricha Filament for 3D printing $ 27 Personal / Reimbursed Raspberry Pi Camera $ 20 Dr. Pasricha 16 GB Micro SD Card (x2) $ 28 Dr. Pasricha Power bank battery $ 17 Dr. Pasricha Tripod for phone mount $ 30 Personal / Reimbursed Total Spent $ 402 Table 3: Actual Budget 16

18 Appendix C (Timelines and Deliverable Documents): Deliverables/Objective Dates Team Members Details First team meeting 8/30/17 All Team Lead and Job Assignment 9/6/17 Connor for lead Computer Vision calibration 9/15/17 Connor App started 9/20/17 Ben Settled on robot chassis and ordered 9/29/17 All Robot Arrived 10/5/17 All Initial Calibration 10/8/17 All 3D prototyping start 10/11/17 Tyler Changing teams/responsibilities 10/18/17 All Robot command system 10/18/17 Tyler, Aubrey Consolidation of App team 10/18/17 Ben, Tyler Prototype app due 10/25/17 Ben Line following prototype 10/25/17 Connor Roomba manual control 10/25/17 Tyler, Aubrey First Functional Prototype 11/15/17 All Refined Prototype Built 12/7/17 All Scope for next semester Table 4: Project Timeline 17

19 Appendix D (irobot Create 2 Op Codes): Table 5: Create 2 Op Codes 18

20 Appendix E (Sample Codes): Code Sample 1: Color Finding Thread 19

21 Code Sample 2:Line Finding and Command Sending Code Sample 20

22 Code Sample 3: Movement Control Sample Code 21

23 22

24 Code Sample 4: Phone Application Acknowledgements: We would like to acknowledge the help given to us by our graduate student mentor Saideep Tiku for keeping us on track and making sure we always kept the big picture in mind, as well as professor Sudeep Pasricha for allowing us to work on such an amazing and interesting project. 23