Multidisciplinary Senior Design Conference Kate Gleason College of Engineering Rochester, New York 14623 Project Number: P14546 VIRTUAL REALITY DATA GLOVE Corey Rothfuss Project Lead Josh Horner Mechanical Engineer Kayla King Mechanical Engineer Cody Stevens Electrical Engineer Mathew Nealon Electrical Engineer Ryan Dunn Electrical Engineer Faculty Advisor David Yoon Electrical Engineer Customer Ed Hanzlik Dr. Gabriel Diaz Copyright 2014
CONTENTS Contents... 2 Abstract... 2 Introduction (or background)... 3 Process (or methodology)... 3 The design... 4 Benchmarking... 6 Budget and market analysis... 7 Results and Recommendations... 7 References... 11 ABSTRACT Virtual Reality is used to simulate 3D environments using multiple cameras, sensors, and immersive displays. It is also a growing value to the world of technology and for research. Currently calculating hand movements to the virtual reality environments is used using motion tracking cameras. This can often result in poor data because of positions of the hand that the cameras cannot see. The objective of this project is to create a glove that collects data from the hand's movements without relying on motion capturing cameras where data can be hindered by certain hand movement. The project will focus on providing a functional prototype that is lightweight, durable, inexpensive, and does not interfere with the user's natural movements. This glove, when functioning, should be a strong competitor for the current commercial models of virtual reality gloves.
Page 3 INTRODUCTION (OR BACKGROUND) Virtual reality is an exciting way for a user to experience 3D environments that may be difficult or dangerous to replicate in the real world. Although many people see virtual reality as a fun way for gaming, it also has other applications in research of the human body. Currently there is research being done on relating eye movements with corresponding body movements. The current system for collecting this data uses cameras and sensors to relay the information to the researcher for analysis. This works well for most of the body, but becomes problematic when focusing the cameras' attention on the hands. Due to the many movements of the hands it is hard for the cameras to see all of the sensors they need for accurate data. PROCESS (OR METHODOLOGY) As prescribed by the general senior design procedure, the entire first semester of the two-semester timeline was a planning and design phase. Much of this planning involved input from the customer (namely, Dr. Gabriel Diaz from the Center of Imaging Science) in order to draft a list of customer requirements and draw from that, a list of engineering requirements. Input during this customer phase was also given by our always fearless and optimistic guide, Ed Hanzlik, using his experience with senior design teams and the design process. Table 1 shows the final customer requirements that were reached after multiple iterations. Through multiple meetings and discussions with our customer and guide, clear customer requirements were laid out. The importance category on the table is rated on a 1-3-9 scale with 9 being of the utmost importance and 1 being the least critical. Twenty-four total customer requirements were created, translating into 34 engineering requirements. The engineering requirements, shown in table 2, are technical specifications that need to be met in order to ensure that the customer requirement was met. Table 1 Customer Requirements For example, if the question how was asked about a given customer requirement, the corresponding engineering requirement would provide the answer the how question. For each engineering requirement, a marginal and target goal was established. In order to make sure the engineering requirements are met, a detailed set of tests and test plans were developed. And shown in the engineering requirements excel file located on the EDGE website, http://edge.rit.edu/edge/p14546/public/home. Copyright 2014
Table 2 Engineering Requirements After creating the engineering requirements, potential risks were brainstormed and compiled in a Risk Analysis Table. Analyzing and understanding these potential risks and drafting mitigation strategies are key steps of any project. Since the Data Glove is to be used multiple times by many different people, safety and durability were a key concern for us to evaluate in the risk analysis. In order to assign a more empirical value system to these risks, an importance score was used. An importance score is produced by multiplying the risk s severity by its likelihood of happening. The severity and likelihood score used the same 1-3-9 scale that was used for the customer and engineering requirements. Multiplying across a risk s severity and likelihood gives us the hazard score, which can then be used to determine its relative importance on a more empirical scale than completely relying on assumptions. THE DESIGN The Glove Since the design is focused on the hand and wrist, the main component of the assembly was done by placing peripherals on a glove. The design originally started out as to be a custom created glove, the team decided that it would be best to purchase a baseball glove. This glove not only is of high quality and durability, but could also stretch in the case of larger hand sizes. Thus, we purchased an Adidas baseball glove was purchased that was sized to a medium hand but was found through testing of multiple hand sizes that it worked well with other hand sizes to deliver accurate results. Knuckle Assembly The knuckle assembly was made for a variety of reasons. Placing the flex sensors used for data collection directly on the hand caused multiple issues including but not limited to unsmooth bend radii of the flex sensor, twisting and bending of the flex sensor due to curvature of the fingers, collision between front and back sensor for the two sensors used on each finger, and overall safety of the flex sensors to be more durable. What was created was using a three-knuckle design for each finger which is shown in figure 1 (to the right). The knuckle closest the palm and the knuckle at the end of the finger were created to allow for the sensor to stay in place and the curved top allowed it Figure 1 Knuckle Design
Page 5 to create a smoother bend radius to contribute to more accurate results. The middle knuckle has two slots so the front flex sensor and the back flex sensor did not collide during movement and sliding of the flex sensor. Trying to minimize total size of the glove to eliminate the pieces being cumbersome was a key consideration in the design. All parts were 3D printed using the Brinkman Lab of. Microcontroller and Amplification PCB Assembly The microcontroller and amplification PCB assembly serves to capture, process, and transmit the electrical signals generated by the flex sensors. The Flexpoint sensors serve as varying resistances in a voltage divider. As the sensor bends, the change in resistance results in a change in output voltage. The output voltage is used as the raw electrical signal. To condition the signal, amplification is needed. The printed circuit board (PCB) serves as the electrical groundwork and mounting point for all electrical components. The components are made up of 5 dual operational amplifier packages and resistors. The signal is amplified to utilize the full 0-3.3 VDC input range of the microcontroller. The TM4c123GXL is the microcontroller used in the system. It is capable of sampling up to 11 input signals. For the purposes of this project, only 9 are utilized. The code on the microcontroller translates the measured signals into angles and transmits them to a host computer. To accurately translate voltage to bend angles, a calibration step is needed. The user simply needs to press switch #2 (SW2) on the microcontroller to bring the system into the calibration routine, which is indicated by the LED changing from green to blue. The calibration routine will capture the voltage values associate with each angle captured and interpolate the data. This interpolation will generate a polynomial approximation for the output characteristics of each Flexpoint sensor. Figure 2 PCB Layout Wrist Assembly Since the PCB board, microcontroller, and wiring all needed to be close to the hand for accuracy of the flex sensor, a wrist assembly was made to hold all of the associated accessories necessary for data collection. Figure 3 below shows the front half of the wrist plate, it is about 3 inches wide and extends up to half way up the forearm. The initial PCB board design was larger so a large wrist plate was needed but after iterations it turned out much smaller than expected. Through empirical evidence, it was found that the large wrist plate was more secure when the arm and hand were in motion which was crucial so the wires did not break apart. In figure 3, a slot is shown in the front that allows all the wiring to be relocated into one spot to reduce excess wiring from being exposed outside of the glove. Velcro slots were made and Velcro was used to wrap the arm around the wrist plate. All 3D printing was done through Brinkman Lab at. Copyright 2014
Figure 2 Wrist Assembly BENCHMARKING Research was an important part of this project because this is something that most of us have never came across through school or through work experience. Learning how to be compatible with python so Dr. Diaz can use the data for his VR software and all the associated accessories that would be necessary to construct the data glove. Also, to find a reasonable and feasible requirements based on the budget of other companies and their research capabilities was important. Table 3 below shows the benchmarking analysis of all the companies compared with our engineering goals of the project. Table 3 Benchmarking analysis
Page 7 BUDGET AND MARKET ANALYSIS The design team was granted a budget of $1000, 50 percent of the budget was through the RIT Multidisciplinary Design Fund and the final $500 was generously given to us by our customer, Dr. Diaz through the CIAS department and his startup grant fund to help him with his research. However, due to the fact that the design, build plans, and assembly instructions will be freely available to the public, the goal was to keep the system cost as low as possible. This would increase the number of potential users which the glove can create a variety of other uses as well. Market for Virtual Reality (VR) is slightly less than a 1 billion dollar business and projected to grow with companies such as Oculus and Sony making a large impact lately. Having things such as the data glove will increase help connect the eyes through the VR with the hands making it even more lifelike. Unfortunately for us, competitors are charging tens of thousands of dollars for highly accurate gloves so keeping ours accurate and affordable created quite a challenge. Nearly the entire budget was spent for various items, the most expensive item being the flex sensors and iterations of the PCB boards. The team wanted to make the PCB board correct so in the future; all repairs or improvements were to be low cost and easy to repair. RESULTS AND RECOMMENDATIONS This project was intended to give additional data with the use of active camera markers to give accurate results of finger and hand movements in the use of a Virtual Reality environment. Through feasibility, correlating hand movements were determined to be not a scope of this project and only the finger data was to be recorded. This can also be used for other scenarios. All components were successfully created and connected to the wrist assembly. Figure 3 shows the finished product as a whole assembly. When plugged into a computer, the glove can give real-time results of finger movements through voltage differentials of the sensors. Figure 3 Fully Assembled Glove The flex sensors and lightweight glove assembly provides a cost efficient data glove that can be used if given the proper training and calibration which is provided in an instruction manual. Copyright 2014
Calibration Chart 4000.00 3900.00 3800.00 3700.00 3600.00 3500.00 3400.00 Sensor 1 Sensor 2 Sensor 3 Sensor 4 Sensor 5 Sensor 6 Sensor 7 Sensor 8 Sensor 9 3300.00 3200.00 0 10 20 30 40 50 60 70 80 90 100 Angle Figure 4 The Calibration Chart Figure 4 shows the graph of the results of calibration. Calibration was accomplished by having the subject hold their hand steady at three different fixed angles, 0, 45, and 90. 0 was realized by the subject holding their hand against a flat surface. The other angles were measured with a goniometer.
Page 9 Sensor Number Voltage Calibration Equation Angle Calibration Equation Sensor 1 y = -0.0215x 2 + 6.6139x + 3431.7 X=1/430*(66139-(33886987321-8600000*y)^ (0.5)) Sensor 2 y = -0.0735x 2 + 11.4x + 3221.8 X=(-20/147)*( (2692923-735*y)^(1/2)-570) Sensor 3 y = 0.07x 2-2.3378x + 3561.6 X=-1/700*(11689-(7000000*y-24794567279)^(1/2)) Sensor 4 y = -0.0558x 2 + 9.6141x + 3288.9 X=1/372*(32047-(9183482209-2480000*y)^(1/2)) Sensor 5 y = 0.0397x 2-0.4837x + 3691 X=1/794*((15880000*y-58589683431)^(1/2)+4837) Sensor 6 y = 0.0102x 2 + 1.7944x + 3299.2 X=2/51*((63750*y-205292951)^(1/2)-2243) Sensor 7 y = 0.0449x 2 + 0.0154x + 3555 X=1/449*(-77+(4490000*y-15961944071)^(1/2)) Sensor 8 y = 0.0538x 2-1.5199x + 3295.2 X= (-1/1076)*(15199-(21520000*y- 70681694399)^(1/2)) Sensor 9 y = 0.007x 2 + 6.0172x + 3368.6 X=1/35*((7)^(1/2)*(25000*y-51887593)^(1/2)-15043) Table 4 The Calibration Equations From the calibration chart, a second order polynomial was used to linearize the equations. This is shown in the Voltage Calibration Equation column of Table 4. These equations were then flipped so that an input of the micro controller tick was the input and a bend angle was the output. These new equations are shown in the Angle Calibration Equation column of the same figure. These were what were used to generate the bend angle graphs. It is worth mentioning that the direct equations for sensors 3 and 8 were giving the negative angle of what was expected. To counteract this a negative sign was placed at the beginning of the equation. This, coupled with the varying directions of initial curves imply strongly that the calibration method needs improvement. For testing, the following procedure was utilized. The calibrated subject was asked to have their hand flat for two seconds, grasp the cylinder or sphere in question for two seconds, and then put their hand back to being flat. This process was repeated for three trials per object. The objects included four cylinders ranging from 3.5 cm up to 8.75 cm in diameter, and four spheres ranging in size from a golf ball to a whiffle ball. Table 5 Test Results of Grasping Cylinders Copyright 2014
Figure 5 Voltages of Sensors in Ticks Shown in Table 5 are the results from the largest and smallest cylinders. Sensor 8 was omitted from the 8.75 cm cylinder as the results show that the sensor was likely twisted or kinked, as after the initial grasp the output was stuck at roughly 26. Sensor 3 had similar though less pronounced problem as it briefly dropped to the base voltage when the subject s hand was flattened. The voltages are reported not as true voltages, but rather as the number of ticks given by the micro controller. This was done as there was no conversion factor given by the data sheets and these are the numbers that the end user will be working with if they choose to use the raw data. The ranges of ticks varied from sensor to sensor, but were up to 400 ticks for the 8.75 cm cylinder and up to 600 ticks for the 3.5 cm cylinder. Figure 5 shows the three bends in each trial and their average per sensor in graphical form. The 8.75 cm cylinder graph is missing its last trough as that was sensor 8 and it did not give graphable results. Table 6 Mapping of Test Results Figure 6 Angles of Sensors Each of the data points was converted via their calibration equation in Table 4 to its equivalent angle. These numbers were then processed the same way as the voltage tick levels. For the 8.75 cm cylinder trial, sensors 3 and 8 are still the ones giving less than ideal data. For the 3.5 cm cylinder trial none of the sensors were behaving unexpectedly. For the larger cylinder the highest inaccuracy was in sensor 7 with 7.02, and for the smaller cylinder the largest inaccuracy was in sensor 8 with 5.17. However most of the sensors, with a small sample size of three trials, were within 5 of the average at each constant amount of finger bending. Figure 6 shows the graphical representation equivalent of Figure 5 with the standard deviations represented as error bars on the graph at each point. An
Page 11 important note is that for sensors 2 and 4 (data points 1 and 3) on the 3.5cm cylinder actually exceed the limits of the calibration equation and are slightly skewed by MS Excel s inability to process imaginary numbers. If given more time and resources, more research would be done towards different type of sensors for the glove. The flex sensors from FlexPoint were the most accurate sensors to be used for the budget given. Through looking at the benchmarking table, Cyberglove III, which is the most accurate glove cost around $15,000 to create. So for only a $1000 budget we provided the most accurate results that we could. Also more time to record data and test results for more repeatability results could have provided more accurate results. REFERENCES "Arduino Playground - Python." Arduino Playground - Python. Arduino, n.d. Web. 30 Sept. 2014. Kessler, Drew, Larry Hodges, and Neff Walker. "Evaluation of the CyberGlove as a Whole Hand Input Device." Https://smartech.gatech.edu/bitstream/handle/1853/3550/95-05.pdf;jsessionid=9B43B2C34F54B10130D585D5BD2B6E5C.smart2?sequence=1. Georgia Tech, n.d. Web. 10 Feb. 2014. Klowery. "Sign Language Glove V1." Clloks.com. N.p., n.d. Web. 24 Feb. 2014. "Navigation." Welcome SensorWiki.org. SensorWiki, n.d. Web. 25 Jan. 2014. "SPLINE Interpolation and Approximation of Data." SPLINE. Florida State University, n.d. Web. 15 Oct. 201 Copyright 2014