Design of Android Based Hand Gesture Controlled Robot Using MEMS

Transcription

1 Design of Android Based Hand Gesture Controlled Robot Using MEMS P.Chaitanya Assistant Professor, ECE Dept. Sreyas Institute of Dr.T.Ravichandra Babu Professor, Department of ECE Sreyas Institute of Dr.Syed Basha Shaik Professor, Department of ECE Sreyas Institute of Dr.J.Narendra Babu Professor, Department of ECE Sreyas Institute of Abstract In this paper we propose a frame structure for the implementation of controlling automated machine it turns out to be very hard and cluttered when there comes the piece of controlling it with remote or various switches. A motion Controlled robot is a sort of robot it can be controlled by hand motions not by old catches. Simply need to wear a little transmitting gadget close by which incorporated a speeding up meter. This will transmit a proper order to the robot with the goal that it can do whatever we need. The transmitting gadget included an ADC for simple to computerized transformation and a microcontroller Integrated Circuits which is use to encode the four piece information and after that it will transmit by a ZigBee Transmitter component. At the less than desirable end ZigBee Receiver Components gets the encoded information and translate it by microcontroller Integrated Circuits. This information is prepared by a microcontroller lastly engine driver attempts to control the engines. By including prevention sensor and temperature sensor robot is improved to work in various conditions and environment. Keywords ZigBee Device; Robot;Gadget;Automated, ICs,LPC 2148 I. INTRODUCTION Now a days human-machine connection is moving far from keypad and is turning out to be a great deal more wonderful with the physical world. With every passing day the hole amongst machines and people is being pointed with the acquaintance of new advancements with facilitate the way of life. Innovation has understood an extremely noteworthy part in enhancing the personal satisfaction. One route during which this is done is via mechanizing a few errands utilizing complex rationale to streamline the work. Signals have assumed a key part in lessening this cavity. Currently robots are controlled by remote or mobile phone or by direct wired association. Contemplating cost and required equipment's this things builds the multifaceted nature, particularly for low level application. For instance, in telerobotics, slave robots have been shown to take after the expert's hand movements remotely. Robots are turning out to be progressively valuable on the war zone since they can be furnished and sent into perilous regions to perform basic missions. Controlling robots utilizing customary techniques may not be conceivable amid clandestine or unsafe missions. MEMs based robot was produced for interchanges in these great situations where writing on a console is either illogical or unimaginable. This paper reports an adjustment of this correspondences for transmitting motions to a military robot to control its capacities. Signals were utilized to control a pick and place robot model. This framework can explore the remote robot in the diverse environment utilizing different motions orders. In this framework, the robot works from transmitter area with a decent quality inconstructed accelerometer sensor. Mems sensor is utilized to peruse the position of hand motions to produce summons for the robot and is taken as an info, position of Mems is then used to extricate the signal order. The summon prepared by expert microcontroller and is sent through the ZigBee transmitter. At ZigBee recipient area gets information from slave microcontroller through ZigBee module and as indicated by Mems position; information got the collector side microcontroller prepared to incite the engine by driver circuit associated with ARM7microcontroller. At long last the robot is moved in every single conceivable bearing in the earth utilizing six conceivable sorts of charges which are Forward, Backward, Right, Left and arm developments. Signal charge can have one of the six conceivable orders as indicated Numerous frameworks exist that are utilized for controlling the robot through motions. Some signal acknowledgment frameworks include, versatile shading division, hand finding and marking with blocking, morphological sifting, and afterward motion activities are found by format coordinating and skeletonising. This doesn't give dynamicity to the motion inputs because of layout coordinating. Another framework utilizes machine interface gadget to give continuous signals to the robot. Simple flex sensors are utilized on the hand glove to quantify the finger twisting, likewise hand position and introduction are measured by ultrasonics for signal acknowledgment. The rise of administration robots in mid 90's trailed by the advancement of Natural dialect interface through console has been given by Torrance in In 2008, Chinese movement police framework utilized two 3-hub accelerometers altered ISSN: Page 1

2 on the back of their arms that were synchronized with activity lights. In 2010, Sauvik Das et al have utilized an accelerometer as a potential spying gadget to show areas and exercises of client without one's information. One of the restrictions was that inbuilt accelerometer Smartphone would need to be in the same spot as was in the preparation mode to make exact expectations. In 2010, Smartphone's were utilized to control Universal Robot Control System by the understudies of Kyung pook National University, Korea, to plan an ongoing robot control framework in omnipresent environment. Specialist's proposed vision-based interface that included signal acknowledgment through camera to give geometrical data to the robots. They created versatile robot frameworks that were told through arm position. II. RELATED WORK The appearance of robots can be traced back to the 90 s with Helpmate Robots and Robot-Caddy. Since then, there is an exponential development in the turf of robotics, and controlling robots through human gestures have been the topic of investigation for as long time. With the implementation of gestures to control robots, there have been several methodologies to execute the action. Some of the related works are being described in this section: A. Light-based Gesture Recognition Light tracking and controlling of robots with light sensors are being done in a lot of cases. Such robots are independent in nature. Generally, there are some light sensors connected with the robot. The sensors send some rays of light and track them as they gets immersed in the surface or reflected back to it. According to this, the robot can be line-sensing robots where it is made to follow a black or a white path separately. B. Vision-based Gesture Recognition Several robots are designed to be controlled by vision-based gestures. In such robots are used the sensors for an interface to control the robot with some manipulators. The input gesture can be some patterns, movements of hands, color tracking, face recognition, finger tracking, or some templates. They are also used in ball tracking and Robo-football games where the robots play the traditional game of football by tracking the movement of the ball. Though it has paved a way for advanced robot operations, but it is affected by factors such as illumination, foggy weather, background lights, etc. C. Motion-based Gesture Recognition The motions can be used to control a robot. This is generally done by incorporating an accelerometer to control the robot wirelessly. This can also be done using sensors. This method is beneficial over other methods in the sense that it can interact with machines naturally without being intruded by the factors that affects the mechanical devices [3]. One important development in this field is done by Sauvik Das et al in 2010, where he designed a spying device yielding location and activities of the user without his/her knowledge. D. Sixth Sense Technology The Sixth sense technology begins in 1990 by Steve Mann who implemented a wearable computing device via neck projector or head-mounted projector coupled with a camera. Later, following his idea, Pranav Mistry, a young research scientist at MIT at that time came up with new applications of this technology. Pranav Mistry came up with the name Sixth Sense Technology and has since been named Wear Ur World (WUW). This technology applies all of the techniques mentioned above and designing applications that give an intuitive output with the connection of internet III. TECHNICAL REQUIREMENTS The technical requirements for this system of hard ware and software are as follows: A. LPC2148 MICROCONTROLLER LPC2148 microcontroller board based on a 16-bit/32-bit ARM7TDMI-S CPU with real-time emulation and embedded trace support, that combine microcontrollers with embedded high-speed flash memory ranging from 32 KB to 512 KB. A 128-bit wide memory interface and unique accelerator architecture enable 32-bit code execution at the maximum clock rate. For critical code size applications, the alternative 16-bit Thumb mode reduces code by more than 30% with minimal performance penalty. The meaning of LPC is Low Power Low Cost microcontroller. This is 32 bit microcontroller manufactured by Philips semiconductors (NXP). Due to their tiny size and low power consumption, LPC2148 is ideal for applications where miniaturization is a key requirement, such as access control and point-ofsale. B. Mechanical Componets Motor driver is an additional requirement in this work. But to make the output more intuitive, this component is necessary. DC motors are used in this work to rotate the four wheels due to its effectiveness while designing robot with a high gear ratio. The motor shield though a commercial manufactured motor driver is used to complete the work. C. Bluetooth Module Bluetooth is a global standard for Bluetooth connectivity. This is an essential component in this work. This module connects the microcontroller and ISSN: Page 2

3 the Android Smartphone for data exchange. The module used here is HC-05 Bluetooth module. It is an easy to use Bluetooth SSP with typical -80dBm sensitivity, up to +4dBm RF power, low power 1.8V operation and several software properties that facilitates the connected it. D. Android Smartphone This is the remote components that senses the users hand movements and send its determinant value to the Arduino microcontroller via the Bluetooth module. The Android application designed for this work does the calculation of the determinant. E. Accelerometer Micro-Electro-Mechanical Systems, or MEMS, is a technology that in its most general form can be defined as miniaturized mechanical and electromechanical elements (i.e., devices and structures) that are made using the techniques of microfabrication. The critical physical dimensions of MEMS devices can vary from well below one micron on the lower end of the dimensional spectrum, all the way to several millimeters. Likewise, the types of MEMS devices can vary from relatively simple structures having no moving elements, to extremely complex electromechanical systems with multiple moving elements under the control of integrated microelectronics. The one main criterion of MEMS is that there are at least some elements having some sort of mechanical functionality whether or not these elements can move. The term used to define MEMS varies in different parts of the world. In the United States they are predominantly called MEMS, while in some other parts of the world they are called Microsystems Technology or micromachined devices. While the functional elements of MEMS are miniaturized structures, sensors, actuators, and microelectronics, the most notable (and perhaps most interesting) elements are the microsensors and microactuators. Microsensors and microactuators are appropriately categorized as transducers, which are defined as devices that convert energy from one form to another. In the case of microsensors, the device typically converts a measured mechanical signal into an electrical signal. IV. SYSTEM DESCRIPTION Gesture recognition is the main aim of this work. The user holds an Android operated Smartphone, and moves, or rotates his hand in any direction. The accelerometer within the phone is regulated to generate a maximum and minimum value for the movement of the hand in three dimensional coordinates depending upon the external environmental conditions. The android application does the work of sensing the accelerometer calibration and generating the maximum and minimum values from it. Depending upon the values obtained, it sends a determinant value to the microcontroller using Bluetooth. The Bluetooth module receives data and transmits it to Arduino where it checks the determinant value and moves the robot accordingly. The whole process is under an infinite loop, so it runs as long as the power is supplied. The output depends on the accelerometer inputs directly that can be used to control the robot. The accelerometer input depends on the gestures of the user s hand. The steps stated above are broadly described in this section. The system consists of the following steps to work as mentioned: A. Block Diagram of the system Figure.1(a): Block Diagram of the Transmitter Section Figure.1(b): Block Diagram of the Receiver Section B. Transmission of data from Android application to Motor driver The input to the application is the direction of movement of hand of the user given by the accelerometer. This is analog in nature. It is then digitally coded by the Android application before sending it to Arduino by HC-05 Bluetooth module. The signal goes to the digital pins of the Arduino board, which has an inbuilt AD/DA converter of 8 bit. The Arduino process the received data. Based on the data received, appropriate signal is transmitted to ISSN: Page 3

4 the motor driver to rotate the motor in such a way that the robot moves in the direction of movement of the users hand. application lies between these specified values, then the corresponding determinant is generated. This determinant is sent to the microcontroller, which then receives the determinant value, process it to recognize the corresponding gesture, and sends signals to move the robot accordingly. Figure 2: Transmission of data The template is designed so that author affiliations are not repeated each time for multiple authors of the same affiliation. Please keep your affiliations as succinct as possible (for example, do not differentiate among departments of the same organization). This template was designed for two affiliations. C. Receiving the data The data is received from the Android Smartphone via HC-05 Bluetooth module on the digital pins of the Arduino microcontroller. It is then processed in Arduino. This processed data is received by the Adafruit motor shield. Based on the data. Figure 3: Receiving the data D. Gesture Recognition Android Smartphone s are equipped with inbuilt accelerometers. The application designed in this work retrieves the value of the accelerometer and sends a determinant value to the microcontroller via Bluetooth. As the user moves his hand, the accelerometer reading changes. It is then retrieved by the application. There are two values: One is maximum value and the other is minimum value. The range is specified using these two values for each function of the robot. If the value retrieved by the Figure 4: Gestures for movement of the robot Figure 4 shows the gestures to control the movement of the robot. When the user tilts his hand forward, the gesture is recognized as the forward movement, and the robot moves in the forward direction. The angle of the tilt or the difference between the angle of tilt of users hand and the threshold value of forward movement gesture determines the speed of the robot. When the user tilts his hand on the right direction, the gesture is recognized as the right turn, and the robot moves in the right direction. When the user tilts his hand in the left direction, the gesture is recognized as the left turn, and the robot moves in the right direction. The angle of the tilt of users hand determines whether the left or right turn is a normal turn or a sharp turn. A sharp turn is one in which a car changes direction without slowing down before turning. When the user tilts his hand backwards, the gesture is recognized as the move backward gesture, and the robot moves in the backward direction. If the users hand is somewhere between the two gestures, i.e., the accelerometer value is somewhere between the threshold of two directions(forward and left turn, left turn and backwards, backwards and right turn, forward and right turn), then the robot moves in that diagonal direction. E. Movement of Motors and Wheels There are four DC motors used in the design of this robot: one motor for each wheel. The motors are controlled by the Adafruit motor shield. The shield is stacked on top of Arduino. Every shield stacked can run 4 DC motors. Installing the Adafruit Motor Shield library gives the flexibility of using the motors just by calling some pre-defined functions as motor1.setspeed(value) that sets the speed of the motor to 250 rpm, or motor1.run(forward) that makes the motor1 to rotate forward [10]. These functions are called from the program burnt in the ISSN: Page 4

5 Arduino microcontroller. The signal is sent to the motor shield that runs the motors. The wheels are connected to the motors. 4 DC motors are used Two for left wheels, and two for right wheels. When the signal received in the motor shield is to move forward, all the four wheels of motors rotate forward, this turns all the four wheels in the forward direction. The robot moves in the forward direction. When the signal received in the motor shield is to turn the robot in the forward left direction, the left diagonal motors are rotated backwards while the right diagonal motors are made rotated forwards. This makes the robot turn in the forward left direction. When the signal received in the motor shield is to turn the robot in the forward right direction, the right diagonal motors are rotated backward while the left diagonal motors are rotated forwards. This makes the robot turn in the forward right direction. When the signal in the motor shield is to move backward, both the pairs of the motors are rotated backwards resulting the robot to move backwards. When the signal in the motor shield is to stop the robot, all the motors are made stationary resulting the robot to stop. Similarly, to rotate the robot in backward directions, similar methodology is used. To turn the robot in the backward left direction, the left diagonal motors are rotated forwards while the right diagonal motors are rotated backwards. This makes the robot turn in the backward left direction. To turn the robot in the backward right direction, the right diagonal motors are rotated forwards while the left diagonal motors are rotated backwards. This makes the robot turn in the backward right direction While DET == B, move the robot backwards 4. While DET == L, move the wheels left 5. While DET == R, move the wheels right 6. While DET == I, move the wheels right forward 7. While DET == J, move the wheels right backward 8. While DET == G, move the wheels left forward 9. While DET == H, move the wheels left backward B. Function check() Step 1: Initialize DATAIN Step 2: Step 3: If data on the serial lines > 0 then End If Step 4: Set the velocity based on the data received in the multiples of 25. Set VELOCITY Step 5: Return DET Figure 5: Movement of the Motors and Wheels VI. RESULTS V. ALGORITHMS A. Main Module Step 1: Initialize the frequencies of the motors. Step 2: Initialize SERIAL 9600 Step 3: Set the speed for the motors in rpm. Step 4: While (1) do 1. DET 2. While DET == F, move the robot forward ISSN: Page 5

6 Figure 6: The robot moves in the direction of the movement of the users hand: forward-left Figure 7: The robot moves in the direction of the movement of the users hand: back Figure 8: The robot moves in the direction of the movement of the users hand: forward right. VII. CONCLUSION & FUTURE SCOPE The Gesture controlled robot designed in this work has many future scopes. The robot can be used for surveillance purpose. The robot can be applied in a wheelchair where the wheelchair can be driven by the movements of rider s hand. Wi-Fi can be used for communication instead of Bluetooth to access it from a greater distance. Edge sensors can be incorporated to it to prevent the robot from falling from any surface. Some camera can be installed which can record and send data to the nearby computer or cellphone. It can be implemented on a watch, or in any home appliances like Room heater. Modern ARDUINO chips support Intranet as well as Internet connections which can be utilized to a greater extent. This robotic car can be enhanced to work in the military surveillance where it can be sent to some enemy camps and track it s activities via Internet. With a mind full of creation, the possibilities are endless. In this paper, the design and implementation of Gesture Controlled Robot is presented and developed using Arduino microcontroller and Android Smartphone. An algorithm has been provided and its working is detailed thoroughly. Since the updating possibilities are endless, updating the system has been kept as a future scope. The built device is cheap, and is easy to carry from one place to another. The addition of the some additional sensors or camera will make it more productive. The limitation of the hardware being associated with a system has been reduced to a great extent. As an end thought, the system will allow the user to control it in a way that reduces the gap between the physical world and the digital world with an output more intuitive. ACKNOWLEDGMENT The authors would like thank to the management team of Sreyas Institute of Engineering & Technology, Shri A.Vinay Kumar Reddy (Chairman), Shri Ch.Ravindranath Yadav (Secretary) and Shri A.Hriday Reddy (Vice Chairman) for the invaluable support during the development of this work. REFERENCES 1 Monika Jain, Aditi, Ashwani Lohiya, Mohammad Fahad Khan, Abhishek Maurya Wireless Gesture Control Rbot: An Analysis Available at: upload/december/22-wireless%20gesture.pdf 2 Wang, B., and Yuan, T., Traffic Police Gesture Recognition using Accelerometer, IEEE SENSORS Conference, Lecce- Italy,pp ,Oct Cannan, J. and Hu, H., Human-Machine Interaction (HMI): A Survey.[Online]. Available: 508%20HMI-Survey.pdf 4 Song, M., Kim, B., Ryu, Y., Kim, Y., and Kim, S., A design of real time control robot system using android Smartphone The 7th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI), Busan- Korea, Nov K. Brahmani, K. S. Roy, Mahaboob Ali, April 2013, Arm 7 Based Robotic Arm Control by Electronic Gesture Recognition Unit Using Mems, International Journal of Engineering Trends and Technology, Vol. 4 Issue 4 Available at: V4I4P347.pdf 6 Waldherr, R. Romero and S. Thrun, 2000, A gesture based interface for human-robot interaction, In Autonomous Robots in Springer, vol. 9, Issue 2, pp available at thrun/papers/waldherr.gesturesjournal.pdf. ISSN: Page 6