A Survey on Indoor Object Detection System

Transcription

1 A Survey on Indoor Object Detection System Shalu Gupta & Y. Jayanta Singh Assam Don Bosco University, Guwahati, Assam, India shalu2324@gmail.com, jayanta@dbuniversity@ac.in Abstract - There are some indoor navigational systems for visually impaired people but few can provide dynamic interactions and adaptability to changes. This paper presents an up-to-date survey of indoor navigational techniques i.e. Laser Activated RFID, Smart Indoor Navigation for Visually Impaired, Ontology and Algorithm for Indoor Routing, RFID for indoor environments, NAVI, TANIA, NOPPA, Trekker. Since the beginning of the 1970 s, the research of vision aids for the visually impaired people has been broadly extended. A general comparisons between methodology used in above techniques are also shown. Keywords - Visually Impaired, Indoor technique, RFID tags, Visual clues etc. I. INTRODUCTION A human being acquires information from the outside world mainly rely on their sights. So we can imagine that it is quite difficult for the visually impaired or blind people to live a comfortable life the same as normal human being. In order to help the blind cope with their inconvenient in their daily life, many good scientists spare more effort on the research of vision aids. With the development of advanced human machine interface and effective information processing algorithms and more powerful microprocessors, it is possible to enable the blind to achieve additional perception of the environment and help more to blind people with new technologies. According to WHO (World Health Organization) estimates in June 2012, there are about 285 million people are visually impaired worldwide, out of these 39 million are blind and 246 million have low vision i.e. severe or moderate visual impairment[10]. About 90% of the world s blind people live in developing countries. 65% of visually impaired and 82% of blind people is over age of 50 years, although this group comprises only 20% of the world population. Refractive errors, cataracts and glaucoma are top causes of visual impairment. Cataracts, glaucoma and age related macular degeneration re top causes of blindness. The number of visually impaired from infectious diseases has greatly reduced in the last 20 years. Two types of mobility based devices are established for visually impaired i.e. sensory based vision aid for blind and non sensory based vision aid for blind (RFID Tags, ontology)[14]. Sensory based techniques focuses on without the visual clues, the blind get information from hearing, touching, smelling and feeling. NAVI, voice and SINVI are sensory based techniques. LARFID, RFID and ONALIN are based on electrical simulation to a component of the visual pathway to the blind or visually impaired people. II. REVIEW WORKS a) RFID tags based vision aids for blind RFID stands for Radio Frequency Identification. It is a small radio frequency identification device for identification and tracking [17]. An RFID tag is a microchip combined with antenna in a compact package and it allows to be attached to an object to be tracked. RFID tags are small in size. These RFID tags based devices include door detection, Laser Activated RFID Tags [1], RobotCart [18] etc. b) Laser Activated RFID (2007) Liu and Zhou [1] use a collection of laser-activated RFID (LARFID) tags distributed in the indoor environment. These tags are designed and used as the artificial landmark. Each LARFID tag has a unique ID and its absolute position in an indoor environment will be written into its own memory. Each LARFID tag is represented by its LED, and its position is defined by the position of LED. A tag contains both the distance and bearing information. RFID reader detects all the functioning RFID tags within its reading range. To solve this problem, they add to each active tag a photodiodebased switch circuit which stands between the battery and function part of the RFID tag. A laser pointer with pan/tilt capability will be installed on the mobile Robot. The modified RFID tag which will be activated by the laser beam sent from the mobile robot. Usually the switch circuit on an LARFID tag is off, and the tag is not powered and not functioning. When an LARFID tag is detected and its 17

2 position relative to mobile robot is calculated via stereo vision, the robot shoots a laser beam to the LARFID tag. The laser beam hits the photodiode and generates a voltage signal to open the switch circuit and turn on the power of the tag. Once the LARFID tag is activated, it will be detected by the onboard RFID reader. Then the RFID reader can retrieve the absolute position of the tag, which is stored in the tag memory, by follow up inquiry with the tag beam. Once the laser beam is removed from the photodiode, the switch circuit will be closed, and the tag will be deactivated. Intentionally, only one laser beam is used and each time only one LARFID tag is activated by the laser beam. The mobile robot decides which tag to target, and the RFID reader knows which tag it detects. c) ONALIN (Ontology and Algorithm for Indoor Routing) ONALIN [3] includes concepts applicable for routing and comply with ADA (American Disability Act) standards for building layouts. ONALIN includes three main concepts: Path Element is differentiated into horizontal and vertical, obstacle and landmark. Horizontal paths are those routes on single given floor and vertical paths are those routes that cross from one floor to another. Landmarks are employed to guide users of routes rather than being used to actually calculate routes. Some of the parameters are direction of an escalator, capacity of an elevator and door types. The routing algorithm addresses both feasibility and comfort ability taking into account multiple destinations for both fixed and flexible sequences of intermediate destinations. Once a user request a route from an origin to one or several destinations with their preferred order, the algorithm first retrieves hallway network, and then prunes hallway network based on group under which user s special needs fall. Pruning hallway networks entails removing those inaccessible hallways and POIs (Point of Interest) for underlying group. Once hallway network is pruned, any route traversing network is feasible for that group. To compute a comfortable route, in case of several destinations with specified preferred order, first a comfortable route from origin to first destination, then a comfortable route from first destination to second destination and so on. On the other hand, upon a user request for a route to multiple destinations regardless of their order, except for their last destination, the algorithm computes a comfortable route from origin to each of intermediate destinations. Then it chooses destination with least cost. Then chosen destination will be considered as a new origin and this step repeats until it visits all intermediate destinations. Finally, the last chosen destination is considered as a new origin and a comfortable route is computed from it to final destination. In this users can plan routes in advance. d) RFID for Indoor Environments: It proposes an RFID-based system for navigation which relies on the location information on the tag, a user s destination and the routing server where the shortest route from the user s current location to the destination [4]. The navigation device communicates with the routing server using GPRS networks. The GPS based navigation does not work for indoor navigation. The blind navigation system is composed of three subsystems: the track infrastructure, the navigation device and the navigation server. The track infrastructure consists of RFID tags and each tag is embedded into a stone back and put it on a footpath. The tags store the tag Id and tag location. The navigation device is an embedded system that is equipped with a microprocessor unit (MCU), an RFID reader, a communication module, a user interface module, a memory module. The reader retrieves the information from a tag and transfers it to MCU for further processing then communication module communicates with navigation server to send a request and to receive the planned route from the server for navigation. The navigation server receives tag information of the current location and destination location then calculates shortest path by using the shortest path algorithm, then whole route is returned to the device for navigate. User may navigate outside the designated route, then the device detects the incident according to the location information from the tags along the route and sends a new request back to the server to calculate new route to the same destination based on the new current location information. There are some delay problems in the device which are the communication delay due to the cold start cycle of GPRS modem and the voice delay. III. SENSORY BASED VISION AIDS FOR BLIND The Blind people mainly dependent for getting information upon the hearing, touching, smelling and feeling, hence it is a multidisciplinary task to develop devices for individuals who are challenging in the sight. These visual substitution devices include Electronic Travel Aid (ETA) which is used for mobility navigation, laser Cane[15], NavBelt[14], GuideCane[16], voice[5], NAVI[5] etc. a) SINVI (Smart Indoor Navigation for Visually Impaired): Lee and Leung [2] developed an automated indoor corridor navigation system for blind is called SIN. SIN involves the novel use of only visual clues deciphered from images captured by a camera mounted on its user. 18

3 Visual clues to produce navigation decisions are that it is dynamic. Horizontal Ceiling structures (HCS) give the indoor structure additional support strength. Location of rooms along corridors with these structures can be located by counting the number of HCS detected from certain starting point. In this Passage Direction Edge (PDE) lines are present in almost all building with inbuilt corridors and many things can be deduced from these PDEs. The PDE lines help in detection of end of corridors by getting the PDE lines shorter or disappear, and then this helps in taking next correct action whether to turn left or right. If the PDE lines are discontinued, then there is a passage way extending out from the walkway, which may lead to other destination such as lifts, escalators or another walkway. For detection of door they narrow their scope to T or L shaped handles. The door handle detection algorithm is done in a stepwise manner from left to right, up to down, until a match between the observed and the expected template is found. After finding the door, the last step is to enter and detect n number of HCS to reach destination. When the system is unable to detect certain visual clue which is instructed to, the system will fail. If the system does not detect the required clue then Sin is not able to correctly update its position in the map and it will not be able to decide which clue is next to detected. b) NAVI (Navigation Assistance Visually Impaired) (2003) Liu and Sun [5] use NAVI which includes Single Board Processing System (SBPS) with chassis, digital video camera fixed in headgear, NAVI Vest and stereo earphones etc. the vision sensor acquires the vision information in front of user, the captured image is processed to identify the object in the form of image, the processed image is mapped onto stereo acoustic pattern and conveyed to the stereo earphones. c) Voice VOICE [8] develops a range of devices that translate arbitrary video images from a PC camera into sounds in the hope that users can see with their ears. A device scans each camera snap shot from left to right, while associating height with pitch and brightness with loudness. The voice system is a sonic imaging system for the blind, which scans and digitized the image captured by a video camera, and then digitized image is converted to sound. The x-coordinate of the image is transformed to time, the y-coordinate of the image corresponding to frequency of the sound, and the grey scale of the pixel in the image correspondence to the loudness of the sound. The sound s pitch varies with the position of the pixel on the vertical axis and loudness varies with steps pixel s grey level. d) TANIA Andreas Hub [6] represents the TANIA (Tactile Acoustical Navigation and Information Assistant) system which comprises of a lightweight portable computer suspended in front of the user, with a GPS sensor and small movement sensor mounted on the neck strap. TANIA utilizes an inertial sensor; tablet computer and enhanced mapping to provide precise navigation. The TANIA system allows virtual exploration by tapping on touch screen and real exploration by walking normally. Moving the finger on the map presented on touch screen provides a spatial impression of the current environment or of any other area where adequate digital mapping has been done. In tracking mode, the user s position is automatically centered in the middle of touch screen, where the strips intersect. The user can also receive information about his/her current position. Several soft and hardware keys control the system. Map operation such as scrolling and zooming can be done with the cursor keys. The keyboard can be used to insert augmented text information about the current location and to search for objects and information which is required. TANIA s guiding grid produce a route description. With guiding grids the user is always aware of his or her position relative to map segments, even when physical features of the pathway may have changed since map construction. Guiding grids makes the TANIA system useful in developing countries or regions where there is no street at all. The TANIA system significantly increases orientation and mobility for the blind, deaf blind and visually impaired, especially in unknown environments. The TANIA system does not require alteration of environment, it can be used anywhere in the world where adequate mapping has done. TANIA system is simple, easy to learn, even for elderly people with no computer experience. e) NOPPA (2002) NOPPA [11] system offers an unbroken trip chain for people who use buses, trains or trams in certain area. It combines the internet and user terminal to complete calculations, synthesis commands and create routes. NOPPA is based on personal navigation services. NOPPA [12] includes a speech user interface, a door-todoor guidance system using public transport, GPS functions and several information services. A motorist can use NOPPA to plan routes because it knows the street names and speed limits within Finland. LaureaPOP navigation and guidance system are same to NOPPA, with the exception that access to public transportation and passenger information system. Door to-door requires map data including entrances and continuous guidance about indoor maps and positions which is not generally available [13]. 19

4 IV. ARTIFICIAL FOR VISUALLY IMPAIRED a) Trekker (2003) Trekker is fully portable, resembling a personal digital assistant (PDA). It has talking menus, talking maps and GPS information. It can create maps as well as use information it receives from GPS to navigate the user to a destination. New hardware can be uploaded onto it at anytime to aid the user further. b) Path Planning and Path Following Hua [7] et. al. defines path planning is concerned with finding paths connecting different PoI (Points of Interest) while satisfying certain constraints such as obstacle/hazard avoidance and determining the shortest distance to a destinations. Path following is more concerned with offering sufficiently accurate guidance information to ensure that the user can follow the planned path and dynamically make corrections and updates to the planned route as the user progresses along it. Path planning algorithm introduces the reasoning of indoor environments, intelligent map and path description, and the cactus tree data structure. Indoor environment means detailed indoor infrastructure information: walls and topology, infrastructure and services, features and furnishings. An intelligent map is to standardize the indoor navigation glossary similar to GIS based on GPS for outdoor navigation to assist the path selection, path description and node-to-node guidance using semantic and ontology-based techniques. Cactus tree can be viewed as a consequence of maintaining the relationship between the indoor elements in this intelligent map. Path planning algorithm consists of three parts: cell decomposition, cactus tree based path planning for building floor and area, dijkstra/a* based path planning for within room and cell regions. Path following algorithm considers the characteristics of the positioning and tracking systems, the floor and furnishing structure with the planned path, human factors such as general stride length, average speed, some basic walking habits, the user s profile and their ambulatory movements. The path following algorithm describes an optimal path p which is found from the a-prior knowledge of the in building environment, the distribution of indoor elements with information such as coordinates, shape, contents, which accompany the path p. the user s profile in relation to navigation tasks, such as general speed, general stride length, walk-able arc, navigation node, delay formula, guidance update interval, walking pattern model etc. The performance of a model when guiding a number of visually impaired people is not on real world indoor environments. c) Door Detection Door Detection [9] is a computer vision based system that would use camera vision input and text to speech synthesized output to provide navigation aid. It will be independent from any remote factors and will have inherent characteristic of improvement with further advancements in computer vision. The proposed system will work on the basis of the input and output scheme: input is a sequence of visual images taken per unit time interval of the scene in front of the user. Processing algorithms need to be devised, developed and tested to identify and recognize specific features in the visual scene. Output is a concise informative announcement of the navigational features in the scene, human activity if any, warning of nearby obstacles and potential collisions. Our algorithm makes use of visual information that can be extracted to detect a closed door in front view in a certain range of distance: the door shall have vertical edges on its left and right ends, these two vertical edges shall have opposite intensity gradient with each other and the length of these two vertical edges shall be more or less same, the ratio of length of each vertical edge to the distance between them shall be close to the aspect ratio of door and the upper ends of the two vertical edges shall be more or less at the same vertical level and the lower ends of the two vertical edges shall be more or less at the same vertical level. Some of the related studies are also shown in [20,21,22,23,24,25,26,27,28, 29,30] V. COMPARISON These mobility devices are served as mobility navigation for visually impaired in real time imaging and what type of input required to these devices. Some devices require RFID tags and some other requires visual clues, CCD camera inputs etc. Table 1: Comparison of Mobility approaches and inputs for visually impaired. Mobility Device LARFID SINVI ONALIN RFID voice TANIA NAVI Inputs Laser Activated RFID Tags and stereo vision Visual Clues Ontology RFID Tags and Microprocessor Unit Sonic Imaging System Tapping on Touch Screen CCD Camera 20

5 NOPPA Trekker Door Detection Path Planning Following and Speech user interface CCD Camera Camera vision input CCD Camera VI. CONCLUSION All the above development of recent research of indoor object detection techniques for the visually impaired people tells about the technique used and how these methods are working. VII. ACKNOWLEDGEMENT This survey report is a part of AICTE sponsor project. We deeply acknowledged AICTE (Govt. of India) for sponsoring this research work. VIII. REFERENCES [1] Yu Zhou and Wenfei Liu, Preliminary Research on Indoor Mobile Robot Localization using Laseractivated RFID, in IEEE International Conference on RFID Gaylord Texan Resort, Grapevine, TX, pp , [2] Lee Wee Ching and Maylor K. H. Leung, SINVI: Smart Indoor Navigation for the Visually Impaired, in the 8 th International Conference on Control, Automation, Robotics and Vision Kunming, China, pp , 6-9 th December [3] Patrick M. Dudas, MahsaGhafourian and Hassan A. Karimi, ONALIN: Ontology and Algorithm for Indoor Routing, in 10 th International Conference on Mobile Data Management: System, Services and Middleware, pp , [4] Sakmongkon Chumkamon, Peranitti Tuvaphan thaphiphat, Phongsak Keeratiwintakorn, A Blind Navigation System Using RFID for Indoor Environments, in the proceedings of 5 th International Conference on Electrical Engineering / Electronics, Computers, Telecommunications and Information Technology, pp , [5] Jihong Liu and Xiaoye Sun, A Survey of Vision Aids for the Blind*, in the proceedings of the 6 th world congress on intelligent control and automation, Dalian China, pp , June [6] Andreas Hub Precise Indoor and Outdoor Navigation for the Blind and Visually Impaired Using Augmented Maps and the TANIA System, in the proceedings of the 9 th International Conference on Low Vision, Vision 2008, Montreal, Quebec, Canada, [7] Hua Wu, Alan Marshall and Wai Yu, Path Planning and Following Algorithms in an Indoor Navigation Model for Visually Impaired, in 2 nd International Conference on Internet Monitoring and Protection (ICIMP 2007), [8] Ying Liu, Jean Bacon and Roger Wilson-Hinds, On Smart Care Services: Studies of Visually Impaired Users in Living Contexts, in the proceedings of 1 st International Conference on the Digital Society (ICDS 07), [9] M. Sarfraz and S. M. Ali J. Rizvi, Indoor Navigational Aid System for the visually Impaired, in Geometric Modeling and Imaging (GMAI 07), [10] Blindness and Visual Impairment: global Facts. [11] NOPPA project. Navigation and Guidance for the Blind, archived from projektit/noppa/noppaeng.htm ( ). [12] JyriRajamaki, Petri Viinikainen, Julius Tuomisto, Thomas Sederholm and MiikaSaamanen, LaureaPOP Indoor Navigation Service for Visually Impaired in a WLAN Environment, in the proceedings of 6 th WSEAS International Conference on Electronics, Hardware, Wireless and Optical Communications, Corfu Island, Greece, pp , February [13] Virtanen, Ari, Koskinen Sami, Public Transport Navigation and Information aid for the Visually Impaired, in ASK-IT International Conference, October [14] Jin Liu, Jingbo Liu, LuqiangXu and Weidong Jin, Electronic Travel Aids for Blind Based on Sensory Substitution, in the 5 th international Conference on Computer Science & Education, [15] C. Capelle, C. Trullemans, P. Arno and C. Veraart, A Real-Time Experimental Prototype for Enhancement of Vision Rehabilitation Using Auditory Substitution, in IEEE Transactions Biometric Engineering, Vol. 45, pp , oct [16] I Ulrich, J. Borenstein, The GuideCane applying mobile Robot Technologies to Assist the Visually Impaired, IEEE Transactions on Systems, Man and Cybernetics, pp , [17] RFID, archived from /technologies/rfid/what_is_rfid.asp [18] Vladimir Kulyukin, ChaitanyaGharpure and John Nicholson, RoboCart: Toward Robot-Assisted Navigation of Grocery Stores by the Visually Impaired, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2005), Edmonton, pp , [19] Daniel Keren, Margarita Osadchy, and Craig Gotsman, "Antifaces: A Novel, Fast Method for Image Detection" IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 23, NO. 7, JULY 2001 [20] M. Valera and S.A. Velastin, INTELLIGENT DISTRIBUTED SURVEILLANCE SYSTEMS 21

6 [21] Intelligent distributed surveillance systems: a review, IEE Proc.-Vis. Image Signal Process., Vol. 152, No. 2, April 2005 [22] R. Cucchiara, C. Grana, A. Prati and R. Vezzani, "INTELLIGENT DISTRIBUTED SURVEILLANCE SYSTEMS Computer vision system for in-house video surveillance", IEE Proc.-Vis. Image Signal Process., Vol. 152, No. 2, April 2005 [23] R. Cucchiara, C. Grana, A. Prati, G. Tardini, R. Vezzani "Using computer vision techniques for dangerous situation detection in domotic applications"the Institution of Electrical Engineers. Printed and published by the IEE,2004 [24] Jie Zhou, "Moving Vehicle Detection for Automatic Traffic Monitoring", IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. 56, NO. 1, JANUARY 2007 [25] Junfeng Ge, Yupin Luo, and Gyomei Tei "Real-Time Pedestrian Detection and Tracking at Nighttime for Driver-Assistance Systems", IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 10, NO. 2, JUNE 2009 [26] Raghuraman Gopalan, Tsai Hong, Michael Shneier, and Rama Chellappa "A Learning Approach towards Detection and Tracking of Lane Markings", IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 13, NO. 3, SEPTEMBER