HUMAN MACHINE INTERFACE

Transcription

1 Journal homepage: ISSN: HUMAN MACHINE INTERFACE Priyesh P. Khairnar, Amin G. Wanjara, Rajan Bhosale, S.B. Kamble Dept. of Electronics Engineering,PDEA s COEM Pune, India priyeshk07@gmail.com, aminwanjara@gmail.com, bhosale.rajan@rediffmail.com, kamblesd2013@gmail.com Abstract: We introduce system based on image processing for automatically recognizing a limited set of gestures from hand images for a robot control application. Gesture recognition is already challenging problem. We developed simple, yet effective, procedure for gesture recognition. Our approach contains steps for segmenting the hand region, locating the fingers, and finally classifying the gesture. The algorithm is invariant to translation, rotation, and scale of the hand. We demonstrate the effectiveness of the technique on real imagery. We used Processing C software for IP algorithm implementation. Keywords: IP, Gesture, Segmenting, Processing C 1. INTRODUCTION In this type of interface, gestures are used as commands in the virtual world, or computer games. That is, instead of pressing keys on a keyboard, moving a mouse, or using joystick or trackball, human's body gestures are used to create appropriate commands for a running program. In fact, new proposed interface translates user's gestures into the user's intentions as the input of the system. Translation is either through images, which are motion pictures taken of the user's gesture, or through a wearable device such as a marked glove or a wireless motion transmitter. As an interesting and context sensitive characteristic, a specific gesture may invoke different actions in different environments based on the interpretation defined in the driving engine. For example, turning down of thumb may cause a gun shoot in a FPS game, while it can cause turning right the plane in a Flight simulator. Gesture recognition is a topic in computer science and language technology with the goal of interpreting human gestures via mathematical algorithms. Gestures can originate from any bodily motion or state but commonly originate from the face or hand. Current focuses in the field include emotion recognition from the face and hand gesture recognition. Many approaches have been made using cameras and computer vision algorithms to interpret sign language. However, the identification and recognition of posture, gait, proxemics, and human behaviors is also the subject of gesture recognition techniques. Gesture recognition can be seen as a way for computers to begin to understand human 31 P a g e

2 body language, thus building a richer bridge between machines and humans than primitive text user interfaces or even GUIs (graphical user interfaces), which still limit the majority of inputs to keyboard and mouse. Gesture recognition enables humans to interface with the machine (HMI) and interact naturally without any mechanical devices. Using the concept of gesture recognition, it is possible to point a finger at the computer screen so that the cursor will move accordingly. This could potentially make conventional input devices such as mouse, keyboards and even touch-screens redundant. Gesture recognition can be conducted with techniques from computer vision and image processing. 2. LITERATURE SURVEY Early approaches to the hand gesture recognition problem in a robot control context involved the use of markers on the finger tips. An associated algorithm is used to detect the presence and colour of the markers, through which one can identify which fingers are active in the gesture. To exploit the use of gestures in HCI, it is necessary to provide the means by which they can be interpreted by the computers. The HCI requires static/dynamic movements of human hand as an input. Apart from human hand, HCI can pick up the gesture by the movement of human arm, face or even other parts of the human body. Initially, attempts for this HCI using hand gestures required to use hand gloves. These were known as Glove- Based Devices. Glove based gestural interfaces require the user to wear a cumbersome device and generally carry a load of cables that connect the device to a computer. This hinders the ease and naturalness with which the user can interact with the computer controlled environment. Eventually the awkwardness and unnaturalness of using gloves was overcome with an advent in HCI where the video based noncontact interaction techniques required video cameras or the computer visions as an input device which would take human hand gestures as an input. This vision based non-contact interaction made the gesture inputs more natural. Gesture recognition is a term with a goal of interpreting human gestures through mathematical gestures and mathematical algorithms. Computer vision is the science and technology of machines that is concerned with the theory behind artificial systems that extract information from the images. As a technological discipline, computer vision seeks to apply its theories and models to the construction of computer vision systems. The examples include the controlling processes, detecting events, organizing information, modeling objects or environments and interaction. There is one company named Microsoft, which makes the readymade on system kits for the gesture control, we just have to program the processor used in the kit. Paper titled Recognizing Hand Gestures with Microsoft s Kinect by Matthew Tang explains the idea. Other approach is Gesture Determination for Hand Recognition by Taher KHADHRAOUI. With this approach, it is able to recognize grasp and drop gestures with over 90% accuracy. The paper titled Kinematic Modelling and Manoeuvring of a 5- Axes Articulated Robot Arm by the author T.C.Manjunath gave us the basic knowledge about direct and inverse kinematics of 5 axis articulated robot which is used as a slave in the work carried out. This paper uses the GUI developed in C++ language to perform the successful robotic manipulation task using the developed mathematical kinematic model. 32 P a g e

3 3. METHODOLOGY 3.1 Frame Grabbing National Conference-Ekalavya-2k15, Special issue published by The primary goal of the project is to create a system that can identify human generated gestures and use this information for device control. The user performs a gesture in front of a camera, which is linked to the computer. The picture of the gesture is then processed to identify the gesture indicated by the user. A frame is grabbed from continuous\ stream of images i.e. Video. 3.2 Blurring A Gaussian Blur (also known as Gaussian smoothing) is the result of blurring an image by a Gaussian function. It is a widely used effect in graphics software, typically to reduce image noise and reduce detail. 3.3 RGB to LAB Conversion One problem with the XYZ color system is that colorimetric distances between the individual colors do not correspond to perceived color differences. The CIE solved this problem in 1976 with the development of the three-dimensional Lab color\ space (or CIELAB color space). In this model, the color differences which you perceive correspond to distances when measured colorimetrically. The axis extends from green (-a) to red (+a) and the b axis from blue (-b) to yellow (+b). The brightness (L) increases from the bottom to the top of the three-dimensional model. 3.4 Blob Detection Fig.1: LAB Color Space In the area of computer vision, blob detection refers to visual modules that are aimed at detecting points and/or\ regions in the image that differ in properties like brightness or color compared to the surrounding. There are two main classes of blob detectors (i) differential methods based on derivative expressions and (ii) methods based on local extrema in the intensity landscape. With the more recent terminology used in the field, these operators can also be referred to as interest point operators, or alternatively interest region operators (see also interest point detection and corner detection). There are several motivations for studying and developing blob detectors. 3.5 Centroid Calculation 33 P a g e

4 The Center of Gravity or Center of Mass statistic calculates where the COG of the image lies. The COG is calculated by: COG_X = COG_X + (I*x) COG_Y = COG_Y + (I*y) Total = Total + I, for each pixel where I = (R+G+B)/3 and x,y is the current pixel location. The resulting COG is then divided by the Total value: COG_X = COG_X/Total COG_Y = COG_Y/Total to result in the final x,y location of the COG. Based on the way this COG is calculated brighter pixels will exert more pull on the final COG location than darker pixels. The COG module interface also provides for various graphical overlays. 3.6 Gesture Evaluation Next step after centroid calculation is the Evaluation. Input image that is frame is divided into 5 parts. That s for our convenience. We can divide it into number parts giving possibilities for number of different gestures. To evaluate a gesture, where the centroids are located is the key point. In controller programming, we have given some cases. Centroid locations are related to these. If any gesture that is actually a centroid location fulfills requirements of particular case, the controller executes that particular command. 4. RESULT, CONCLUSION & FUTURE SCOPE Fig.2: Result Window Growing demand for natural Human Machine Interfaces and robot easy programming platforms, a gesture recognition system that allows users to control robotic car was proposed and implemented successfully. Using this system, a non-expert robot programmer can control a robot quickly and in a natural way. The low price and short set-up time are other advantages of the system. The main goal of the project is to wirelessly synchronize the movements of robotic car with the human gestures which was successfully achieved using real time image processing. Gestures are done in front of webcam. These gestures are evaluated and then the output is sent to control robotic car. While scanning a gesture from the beginning, converting it to binary image, thresholding, blob detection, evaluation has been successfully done. Because of the color module that is chosen, appropriate detection & identification of Gesture is 34 P a g e

5 possible. As Processing C has its own in-time compiler, the response time of system is very less. In Future, One can continue simplifying the mechanisms introduced in this project, with the goal of enabling low-cost production of reasonable performance manipulators. We intend to continue exploring low-cost fabrication techniques for functional components, and expect that many, if not all, structural components could be made with the low-cost techniques, allowing fast assembly and further cost reduction. ACKNOWLEGEMENT We are greatly indebted and express profound gratitude towards our honorable guide and co-author Prof. S.B.Kamble who has given us valuable time, motivations suggestions and help throughout the project work right from the selection of topic towards completing the project. She helped us tremendously to overcome shortcoming in our project and prove to be a source of unending inspiration. We are extremely thankful to her. REFERENCES [1]. K.Brahmani, K.S.Roy,Mahaboob Ali Arm 7 Based Robotic Arm Control By Electronic Gesture Recognition Unit Using Mems in International Journal of Engineering Trends and Technology (IJETT) - Volume4Issue4- April 2013 [2]. Meenaakumari.M, M.Muthulakshmi MEMS ACCELEROMETER BASED HAND GESTURE RECOGNITION International Journal of Advanced Research in Computer Engineering & Technology (IJARCET) Volume 2, No 5, May 2013 [3]. Didier Coquin, Eric Benoit, Hideyuki Sawada Gestures Recognition Based on the Fusion of Hand Positioning and Arm Gestures Journal of Robotics and Mechatronics Vol.18 No.6, 2006 [4]. Abdullah Shaikh, GandharKhaladkar, RhutujaJage, Tripti Pathak JavedTaili Robotic arm movements wirelessly synchronized with human arm movements using real time image processing in 2013 Texas Instruments India Educators' Conference, 2013 [5]. AsanterabiMalima, ErolÖzgür, and MüjdatÇetin A FAST ALGORITHM FOR VISION-BASED HAND GESTURE RECOGNITION FOR ROBOT CONTROL, Faculty of Engineering and Natural Sciences, SabancıUniversity, Tuzla, İstanbul, Turkey. [6]. Michael Tornow, Ayoub Al-Hamadi and Vinzenz Borrmann Gestic-Based Human Machine Interface for Robot Control in 2013 IEEE International Conference on Systems, Man, and Cybernetics [7]. AsanterabiMalima, ErolÖzgür, and MüjdatÇetin A FAST ALGORITHM FOR VISION-BASED HAND GESTURE RECOGNITION FOR ROBOT CONTROL Faculty of Engineering and Natural Sciences, Sabancı University, Tuzla, İstanbul, Turkey [8]. Matthew Tang Recognizing Hand Gestures with Microsoft s Kinect Department of Electrical Engineering, Stanford University [9]. Jens Lambrecht and J orgkr uger Spatial Programming for Industrial Robots based on Gestures and Augmented Reality in 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems October 7-12, [10]. Deepali N. Kakade Prof. Dr. J.S. Chitode Dynamic Hand Gesture Recognition : A Literature Review in International Journal of Engineering Research & Technology (IJERT) Vol. 1 Issue 9, November P a g e