CHAPTER 2 LITERATURE SURVEY

Transcription

1 CHAPTER 2 LITERATURE SURVEY Study on Hand Gesture Segmentation detects the hand gesture from a monocular video image under complex background using a skin colour detection and background subtraction algorithm. The arm removal and the noise reduction have improved performance of the gesture recognition. [Bao Hong, 2010: 1-4] Bare Hand Gesture Recognition with a Single Colour Camera has devised a technique for detecting the bare hand without any colour cap. The RGB image is transformed into Hue image. The skin segmentation algorithm is used to localize and segment the hand derived from image and eliminates arm region through using wrist line. Features are extracted to indentify the different hand gesture. [Yishen Xu, 2009: 1-4]. Real-time hand gesture detection and recognition, using boosted classifiers and active learning, proposed a robust, real time method for segmenting and hand gesture identification under dynamic environment hand gesture. To detect and recognize hand gestures extraction of the hand with skin colour detection and adaptive skin colour model, tracing of the hand location based on colour and finally to classify the gestures the boosted classifiers and multi gesture classification tree are used. The classifiers are trained using active learning and bootstrap training techniques, enhancing system performance regarding rate of detection, processing times, and false positives volume. [H. Francke, 2007: ] Adaptive skin colour model for hand segmentation, has executed hand segmentation using adaptive skin colour model. RGB image is changed toycbcr colour space and then CbCR space is mapped into CbCr plane to get cluster of skin region. For classification edge detection is applied to the clusters. [Ahmad Yahya, 2010: ] A Robust Method for Hand Gesture Segmentation and Recognition Using Forward Spotting Scheme in Conditional Random Fields article has put forwarded a forward 5

2 spotting method for segmentation and recognition simultaneously with less time delay. A stochastic non-gesture framework utilizing CRF is utilized to identify gestures without requiring data training. The model yields appropriate confidence measures utilized as part of an adaptive threshold to elaborate the initiation and conclusion of effective measures.[ Mahmoud Elmezain, 2010: ] In Hand Gesture Recognition based on Shape Parameters, the approach is based on shape parameters. The shape information is extracted using computer vision techniques. Using the shape based approach hand gestures is matched. [Meenakshi Panwar, 2012: 1-4] Hand Segmentation using Skin colour and Background information, proposed a new procedure for segmentation of human hands through adaptive skin colour model and background data with effect to split the image into upturned side to improve performance. [Wai wang, 2012: ] Stability analysis for tactile pattern based recognition system for hand gestures, has developed a new pattern recognition system for the artificial limb control using the tactile pattern generated in the prosthetic socket due to muscle contraction. [Yuichiro Honda, 2007: ]. Ethiopian Sign Language Recognition Using Artificial Neural Network, discusses about hand gesture detection and recognition for an Ethiopian sign language by using Garbor filters and PCA for feature extraction and artificial Neural Network for classification of the patterns drawn by the hand. [Yonas Fantahun Admasu, 2010: ] Accurate Recognition of Large Number of Hand Gestures, has established a hierarchical gesture algorithm to recognize the hand gestures. A very low resolution images are processed. Each input frame of sequence is mapped onto a sequence of resulting in a trajectory. Using PCA, HMM and Graph matching, the reduction in the size of the image and noise removal, for temporal analysis and recognition of gestures are made respectively. [A. Shamaie, 2003] 6

3 In this paper Gesture Recognition for Human-Computer Interaction Using Neural Networks, the process incorporates detection and segmentation, Feature extraction and finally recognition of gesture assigned as control commands. Each stage includes a neural network for skin colour detection, principal component analysis and the clustering encoding for hand gestures. [Y. Zhang, 2001: ] Recognizing action at a distance focuses on identifying people s action at a far of distance by using new motion identification based over optical flow assessments under spatiotemporal volume for all stable human image, and identical measure to the closest- neighbour structure. Here the optical flow is regarded as a pattern of noise in spatial domain which forms the motion descriptor in spatio- temporal domain. Nearest neighbour technique is used to classify the actions performed by the person. [A. Efros, 2003: ] Actions as space-time shapes, have considered human interactions in 3D figures developed through outlines towards the time and space volume. The solution characteristic of Poisson equation is utilized for the extraction of attributes i.e. shape structure and orientation, action dynamics, local time space saliency. This method is fast; video alignment is not required and is applicable to images with known background. [M. Blank, 2005: ] Segmentation is a procedure of splitting an image into numerous parts. Automatic segmentation of object of interest from its background in video sequences has drawn lot of interest in computer vision. Skin colour is used for detection, recognition and tracking of hand. A review of various skin colour modelling and recognition techniques along with their numerical evaluation results are discussed in A Survey on Pixel based Skin colour Detection Techniques. [V Sazonova, 2003: 85-92] A real time Face Tracker, discusses about tracking the face of person while moving freely. The system is described as three models. Stochastic model is used to characterize skin colour distribution of human faces. The features extracted from this model are used for tracking the face at different angles and posses and can adapt for any races of human being and for any lighting conditions. The motion based framework is utilized for assessing image motion and to predict the window of search. 7

4 The camera framework it utilized to forecast and balance out the motion of the camera. This procedure is implementable towards recognition of hand gestures. [Jie Yang, 1996: ] Hand Gesture Segmentation Based on Improved Kalman Filter and TSL Skin Colour Model ponders on the new concept of segmentation which avoids the misclassification of objects with the same skin colour in the environment. A new segmentation concept combined with kalman filtering and TSL colour space provides high accuracy and efficiency. Kalman filter assist in anticipating the hand location so as avoid the interference of non hand objects of the skin colour. TSL colour model is used to realize the hand region. Morphological executions are utilized for the removal of outlying objects and holes. [Shu Mo, 2011: ] A New Method for Hand Segmentation Using Free-form Skin Colour Model, deals with the extraction of hand pixel and characterized in YCbCr colour space. CbCr space is then transformed into CbCr plane to form the clusters region of skin colour. Edge detection is applied to form free form skin colour model. The concept works well in complex background. [I Abdullah, 2010: ] Statistical colour models with application to skin detection focuses on the simple histogram learning techniques to build a model for low image characteristics, i.e. colour of the skin. Skin pixel detector is utilized for isolating skin pixels from the contrary using histogram models and its performance is compared with mixture models in skin detection. The histogram model proved more efficient in skin detection. [M. J. Jones, 2002: 81-96] The paper, Hand Segmentation for Augmented Reality System, focuses towards appearance and colour based procedures for detection of the skin for segmenting the hand and the background. The method does not emphasize on the still camera and a uniform background. It does not depend upon the intensity variation of the illumination. The performance is high as the appearance model avoids the misclassification of the pixels of the skin. [Wu Yueming, 2007: ] 8

5 The hand region is recovered using the 3D reconstruction technologies in the paper Estimation of the fundamental matrix from un-calibrated stereo hand images for 3D hand gesture recognition,. A method is presented for predicting epipolar geometry amidst two non-calibrated cameras through stereo-based hand images. The hand undergoes segmentation utilizing RCE neural network based over colour algorithm segmentation and finger tips are extracted as features of interest and are matched with topological features of hand. The procedure has been proven over images based from both calibrated and non-calibrated cameras [Yin X, 2002: ]. A learning based chromatic matching method is used to find out the skin chroma without the effect of illumination. Coarseness of skin is used to segment the skin accurately. To further increase the accuracy of the segmentation low level geometrical constraints is used. [J-S Lee, 2007: ]. Face Detection using Quantised skin colour regions merging and wavelet packet analysis deals with the detection of human faces in the colour images in a complex background with varying illumination conditions. YCbCr and HSV colour models are applied to form the cluster region of the skin. The face skin quality is assessed using wavelet packet decomposition in the detection of faces. The wavelet coefficients of the band filtered images characterize the face skin texture and the statistical differences extracted results in the feature vectors. The probabilistic metric based on the distance parameter is used for classification of the feature vectors as skin or non skin areas of the face using the trained data set. [Arica C, 1999: ] Enhanced Skin Detection Technique Using Block Matching highlights on a technique to detect the skin region in a static image. This procedure utilizes a model encompassing five faces detection with critical alterations to enhance facial detection rates. The hybrid model detects the face region using the block matching technique. [Kaabneh K, 2007: 21-24] Computer monitoring and control with hand movements, implemented computer controlling and monitoring utilizing hand motions through image and computer vision techniques In proposed study, the face of the user is detected with Harr Classifier and the hand region is detected using YCbCr model. The hand is traced by means Blob 9

6 Tracking Technique and the features are extracted using Convexity Defect-Hull. Finally, position of mouse is controlled according to features of hand region, and Computer controlling and monitoring using hand movements are implemented by tracking hand gestures. [R.O. Dogan, 2014: ] Hand gesture recognition with depth images, provides an extensive review of the literature based on the usage of depth assessment and identification of gestures. It concentrates towards depth-oriented gesture identification with reference to the methods developed for hand detection and classification of gestures, the implementation of gesture identification and implementation of OpenNI and Kinect software archives over gesture identification. Studies focuses towards OpenNI and Kinect archives for identification of hand gestures exhibit tendency to focus more towards application, as compared to classification or localization procedures, illustrating that OpenNI tracking procedure is the most efficient to the tested scenarios till now. Regardless, limitation factors for Kinect and associated sensors for gesture identification have not yet been assessed over difficult settings and applications [J. Suarez, 2012: ]. The executed sign, utilizing glove-worn hands, are illustrated as letters over the screen. The framework is built to allow communication between the receiver and transmitter so that the elaboration of the message is conduced accurately and is fully comprehensible. [El Hayek, 2014: ]. A molecular modelling for understanding and visualizing chemical structures in 3D spatial domain and in their dynamic behaviour, using an Augmented Reality frameworks for viable interaction with ocular indicator using molecules. When individuals bring forward multiple molecules within single vicinity, prospective bonds are illustrated and molecules uniquely alter their 3D makeup in accordance with the prospective chemical reactions. 2 Gesture-based procedures, proximity and shake based procedures are recommended to overcome the issues potentially arising when individuals are required to choose one specific bond from amongst a diverse array of prospective bonds whilst utilizing both hands for the alteration of molecules [Maier, Patrick, 2010: 83 90]. 10

7 Author taking into consideration the identical traits found in human hands, i.e. featuring one thumb with four additional fingers, has written Hand gesture recognition based on shape parameters," with the fundamental goal to provide a realtime framework for identification of hand-based gestures over viable shapes, i.e. finger and thumb status, orientation, mass centre etcetera. The framework presented in this research is based entirely on shape-related factors with the hand committing the gestures. The input sequence of images obtained from a simple web camera with 20 fps and seven mega pixel intensity is pre-processed to remove background noise. K- mean clustering is utilized for segmenting hand from the background. [M. Panwar 2012: 1 6]. A Kinect based dynamic gesture trajectory identification method which overcomes the limitation of gesture recognition rate in traditional dynamic gesture track recognition system uses OpenNI, under which palm has been elaborated as Kinect node. The palm position can be quickly and accurately obtained. The HMM technology is used for training the gesture models, therefore leading to enhanced rate of detection. The results demonstrate that these frameworks are faster and efficient with identification rates as compared to the conventional frameworks. [Youwen Wang, 2012: ]. Real-Time Hand Gesture Detection and Recognition Using Bag-of-Features and Support Vector Machine Techniques," presents a hand gesture recognition model that detects and tracks the naked hand under a busy background using detection of skin and position of the hand s outline similarity procedure after deletion of the face. The gestures are recognized using BoF and multi-class Support vector machine. The generated gesture commands are employed to control an application. The features from every trained image are extracted by applying SIFT. A vector quantization procedure mapping features belonging to all trained images into a solitary dimensional histogram vector (BoW) after K-means clustering. The SVM is fed with the histogram to construct a trained classifier. In each input image frame the hand is detected using the algorithm, followed by the feature extraction for all smaller images featuring solely the observed gestures. The features extracted are entered into group systems to categorize them in a BoW format, which is eventually applied to the multiclass SVM to recognize the action of the hand [Dardas, N.H, 2011: ]. 11

8 A robust hand gesture detection algorithm based on DWT and multi- class probability is proposed. Directional features based on Quaternion algebra are extorted utilizing a colour-depth camera. Directional characteristics, i.e. orientation or position which are invariant are utilized. DWT is applied to make speed and size of gesture as invariant, alongside improving the detection capacity. Hierarchical thresholding for deforming distance and gesture probability are utilized for gesture detection. Multi-class probability estimates are employed for classification of gestures. [Pisharady, P.K, 2013: 30-36] A real time model to identify the action of the hands based on wave controller technology is used to simplify the interaction between the human and machines. The presently interfacing devices between the human and computers like mouse, joystick etc suffers from major drawback that have restricted the capability of users, as they are required to respond to computer through the physical contact. Voice recognition cannot be applied where multiple users are present. These mentioned disadvantages are overcome by using hand gesture identification framework. It is also to be extended to the sign language recognition [Zhengmao Zou, 2010: ]. A n 6 of liberty over virtual mouse hand gestures utilizing a webcam powered through USB is focused in this article. The procedures of hand gesture and tracking constitute the skin colour, finger movement and hand motion. People with disabilities are able to access the computer easily. [Xingfeng Wang, 2010: ]. An efficacious skin colour detection system utilizing face detection technology to enhance skin model for a précised individual, whilst tracking is gained through a basic data algorithm is proposed. An efficient gesture identification procedure utilizing various extracted data, i.e. hand motion, local features, finger tips, and hand orientation is brought forward. The suggested framework for identification of hand gesture is implemented towards administration of computer mouse and executing computer applications and games [Nam Vo, 2010: 1-6]. A Real Time Vision-Based Hand Gesture Interaction" has focused on vision based gesture recognition. This application necessitates pinpoint identification and detection. Moreover, the clutter and real environment make a significant influence 12

9 towards the recognition procedure. In this study, a real-time, vision-oriented, hand motion identification system was suggested for administering the desktop cursor, allowing for the navigation through utilizing only gestures of the hand [Yee Yong Pang, 2010: ] A system was proposed for physically challenged people who are powerless to use conventional input systems for communication with computers. In this paper, Computer commands are implemented through hand motion. Dynamic and static movement of hand are obtained by using inertial navigation sensor to administer the mouse cursor, allowing for the opening and usage of computer through motion. Therefore accelerometer profiles are transformed into interactions without wired connection. The tool includes nontactile usage of equipments to alter and administer them as per the provided hand motions [Dand D, 2010: 55-60].]. Advanced mouse pointer control using trajectory-based gesture recognition presents a technique for the administration of the mouse pointer motion utilizing basic gestures from the human hand, utilizing a camera. A real-time algorithm for tracking is implemented over adaptive skin identification and motion assessment. The motion and steering of movement is assessed and then utilized for the identification of a gesture. A section of interest algorithm is suggested, for scaling motion when users are stationed at a far distance from camera. The steering of the pointer is scaled subsequently. [Manchanda K, 2010: ]. A realistic game system employing a multi-modal user interface is proposed in A realistic game system using multi-modal user interfaces," The framework utilizes gaze tracking, a hand gesture identification and analysis of bio-signal. In this system, addictive and realism-orientated graphical software is implemented over the Head Mounted Display with an eye ball tracer, hand gesture recognizer and a bio-signal analysis. The grasping and pitching towards a object are executed through the individual s gestures by means of a Hand glove. Based on the measurement and analysis of bio-signals of the user are used to control the difficulty adaptively. [Hwan Heo, 2010 : ] 13

10 A Real-Time Hand Gesture Recognition System for Daily Information Retrieval from Internet", proposed a system to acquire regular data through the internet, i.e. news, finance news, weather information without requiring interaction with a mouse or keyboard. The daily information s are regained from Internet, using his/her hands' movements. On selection of the function by means of hand gestures the system gives out the information to the users through synthesized speech. The system also provides individual services by identifying the user with the face detection technique. Under this study, a PCA technique is utilized for the identification of facial and hand gestures, and afterwards, gestures are registered into the database for their meaning for further practical applications [Sheng-Yu, 2011: ]. In this paper, the hand gestures captured from the camera provides virtual reality mechanism which can be operated on the entire system in real time. Hand gesture recognition system is composed of three steps namely skin detection, feature extraction and recognition A virtual mouse is designed for moving the cursor up and down [Aksac A 2011: ]. The paper elaborates regarding the virtual touchpad, one that is built utilizing a camera and some other identification procedures, generally hand-based. It utilizes a section of a rectangular object as a pad, functioning similar to a conventional touchpad but features an additional array of controls. The pad supports different clicks, and allowing for gesture-based commands based on the hand and finger motions. This pad is capable of identifying and commanding most simple interactions, i.e. V shape, horizontal, vertical etcetera [Edwin G, 2011: 1-5]. Gesture Recognition Based Mouse Events describes an efficient strategy for mouse points and executes several basic operations, i.e. double and single clicking, dragging etc. utilizing identification procedures. Identifying motions can become a difficult tasks, as there are several aspects involved, e.g. modelling of motions, analysis, recognition of patterns, and learning according to machines. Taking into consideration all the critical factors, a framework has been established with the capacity to identify the motion of hands and fingers, fully understanding the meaning behind it. Coloured caps are utilizes in place of fingers to identify different factors, i.e. skin and background colour. Therefore, identifying the gestures, different events be performed 14

11 using the cursor. This software was created through MATLAB, and Windows is the supported OS [Rachit Puri]. The subjects like the fast response time and high-level precision of the systems are conferred with a stress on computer vision techniques, as gesture interaction presents technical problems. This paper presents an updating study of real time applications of the system using hand gesture along with body movements, vision, facial expression and voice recognition in health clinics and supportive technologies, entertainment, disaster management and in robotics. [J. P. Wachs, 2011:60-71] Model-based 3d hand posture estimation from a single 2d image, has focused on three dimensional hand gesture detection based on a two dimensional image. The hand prototype with 27º of freedom is introduced and its limitations are examined to reduce the 27º to 12º of freedom without any major deterioration of performance. A The two dimensional feature points are used to estimate three dimensional hand poses through the algorithm [C. Chua, 2002: ]. The study elaborates on developing efficient gestures of the hand identification framework through the Kinect sensor. To acquire shape of hand acquired from the sensor, FEMD procedure is implemented. FEMD takes into consideration only the fingers, and disregards fingers. Therefore, allowing for the ascertaining of hand shape featuring trivial discrepancies. The long and expansive study illustrates the efficacy, precision, and effectiveness of the gesture identification system [Z. Ren, 2011: ] Hand gesture estimation and model refinement using monocular camera-ambiguity limitation by inequality constraints, has proposed a method to find concurrently the shape and gesture of hand from the monocular imaging system without the depth information. The crude gesture is detected by matching the shape of the hand. Using motion detection and shape features the Region of Interest is efficiently reduced and possible crude gestures are developed. The crude gestures with high probability are selected and corresponding features are extracted. The modified Extended Kalman Filter with condensed distribution is used for fine-tuning of a three dimensional shape prototype and the crude gestures by disparity limitations. The depth uncertainty is 15

12 restricted with the knowledgeable observations of the images. The ambiguity of the balanced gestures are determined by producing and conserving the several solutions [N. Shimada, 1998: ] A frame work is designed for Bayesian tracking to approximate the posterior distribution at multiple resolutions. To remove least probability cluster and to estimate the distribution to a subjective accuracy, a tree based delineation of distribution is projected in this paper. In tree based depiction, the leaves describe the separation of the Region of Interest with the steady density. This method is used to track three dimensional distinct and flexible motion in a complex environment. The joint angles and the orientation of the hand are estimated in this method. [B. Stenger, ] The paper compares the performance of tracking system in kinect sensor camera and the optitrack in optical system in this paper. The data accumulated from the six upper limb Data is used in game based rehabilitation application. The Kinect and optitrack showed equal performance in tracking the movement of the hand. Kinect provides an prevalent accessibility for patients to recover from paralysis stroke and muscle rejuvenation in clinic and home environment.[ C Y Chang, 2012: ] 16