Development of Pick Up Sticks Game in a 3D Virtual Environment Using Leap Motion

Transcription

1 UTM Computing Proceedings Innovations in Computing Technology and Applications Volume 2 Year: 2017 ISBN: Development of Pick Up Sticks Game in a 3D Virtual Environment Using Leap Motion Ikmal Faiq Albakri 1 and Norhaida Mohd Suaib 2 1,2 Faculty of Computing, Universiti Teknologi Malaysia (UTM), 81310, Johor Bharu, Johor, Malaysia { 1 ikmalfaiq@yahoo.com, 2 haida@utm.my} Abstract. This paper is about a project using Leap Motion technology in a 3D virtual environment Pick Up Sticks game. The main objective of the game is to be a platform for creating a creative mindset of a young children and preserving the urge feeling of realism while playing the 3D virtual environment Pick Up Sticks. By using Leap Motion controller, it is hoped to encourage more developers to contribute to the rising of edutainment games for people to enjoy. Therefore this project developed a prototype of 3D virtual Pick Up Sticks game using Leap Motion technology. The hand gesture interaction approach is being developed and integrated with the 3D Pick Up Sticks game prototype using Unity3D engine. Finally, the 3D Pick Up Sticks game prototype has been evaluated by using some testing method such as Alpha testing and Beta testing to make sure the game can be functioning well as expected. Keywords: Leap Motion, hand gesture, Pick Up Sticks game, children, 3D virtual environment. 1.0 Introduction The traditional Pick Up Sticks game is now forgotten because the rise of modern technologies. Children tend to spend their time using their notebook or mobile phones to play games in a virtual life [1]. Besides, the bookstores do not sell the sticks anymore and if they do, children cannot keep it safely because the sticks are easily to be lost. There are some initiatives to recreate the Pick Up Sticks game. For example, the Pick Up Sticks game in mobile applications that can be found in Google Store [2]. With some changes in the gameplay, it is safe to say that the concept of the game can be preserve and maintain. However, the game is created in 2D environment. Thus making the urge of playing in a natural feeling cannot be felt. And somehow, it will not make the game interesting as it used to be. Therefore, the aim of this project is to develop a prototype of virtual Pick Up Sticks game. In this projects, there are three objectives that must be achieved. The first one is to design a virtual Pick Up Sticks game, the second is to develop and implement hand gesture interaction using Leap Motion device into the 3D Pick Up Sticks game and the last one is to test the 3D Pick Up Sticks game in a Windows PC. This paper is organized as follows: In section 1, the introduction of the paper is presented.next, in section 2, the literature review of the related works is discussed. While in section 3, the project methodology of the system is explained. Then, the implementation of the project is described in section 4. Then, the test and results conducted being analyze in

2 2 section 5. Finally, the conclusion and future works in enhancing the project is elucidated in section Literature Review This section will discuss about the analysis and literature review being done related to the project. 2.1 Pick Up Sticks Game Pick up Sticks has been created for hundreds of years, but it is such an old game that many people do not know how to play anymore [3]. Also known as jackstraws or spillikins, the game is played by both adults and children in order to strengthen their physical and mental skill. Using a bundle of sticks, between 8 and 20 centimeters long created from wood, straw or plastic, the game needed a lot of concentration from the players in order to win the game. To play Pick Up Sticks, a selected player would need to dump the different colors of sticks on any large, flat surface such as table or floor. Then, when the sticks is scatter, the players would take turns to pick up the sticks, one by one without disturbing the rest of the sticks in the pile. If another stick is moved, the player will forfeited his turn and the next player would take a shot at it. The process will continued until all the sticks were picked up by the players. Lastly all the points were added, the player who get the highest score will win the game. 2.2 Application of Leap Motion Technology Just as big as 3 inches long and 1-inch wide, the Leap Motion controller can detect roughly about 1 meter of hemispherical area. Even though, the number of Leap Motion games or applications is very limited right now. The Leap Motion is a glimpse of future. There are already some applications that display some promising potential of the technology in different specific fields such as medical [4], 3D modelling [5] and astronomy. Without any doubt, Leap Motion will one day promised to alter how human think of using computers. 3.0 Project Methodology In order to construct collaborative Hand Gesture Recognition using Leap Motion device in Pick Up Sticks game, virtual environment need to be mixed with some of the concepts from the real world. Therefore, several phases of project development are planned which are: Phase 1. Preliminary investigation and data collection of Hand Gesture Recognition for Pick Up Stick Game. In this phase, research focused on exploring prior approaches of Hand Gesture Recognition for Pick Up Sticks game including their technique to achieve the seamlessness and enhancing reality in the virtual environment.

3 3 Phase 2. Design the well-suited Hand Gesture Recognition technique in the Pick Up Sticks Game. Based on the information explored and gathered in the previous phase, a design phase of collaborative Leap Motion Technique for interaction to effectively overcome the major problems addressed in real Pick Up Sticks game being made. Phase 3. Develop collaborative Hand Gesture Recognition prototype for Pick Up Sticks Game. After the design phase is completed, the frameworks of collaborative Leap Motion for representing the seamlessness and enhancing reality in virtual environment for Pick Up Sticks game is setting up. Phase 4. Integrate Hand Gesture Recognition technique with the Pick Up Sticks Game. Then, the Hand Gesture Recognition technique in collaborative with Leap Motion is developed to create a virtual 3D Pick Up Sticks Game in order to give a whole new level of playing the game to the world. Phase 5. Testing and evaluation This phase will test and evaluate the Leap Motion Interaction technique in Pick Up Sticks Game. 4.0 Implementation The game is implemented based on the methodology discussed in previous section. As mentioned before, the game mechanics requires the integration of the Leap Motion controller with Unity 3D in order to actualize the physical interaction between the input and the objects in the 3D environment. 4.1 Implementing Sensor-Based Tracking System As the 3D gesture inputs are captured from the Leap Motion, the hand gesture recognition device is setting up perfectly at the front of the Windows PC. The sensory space of the Leap Motion is placed in recommended range without disturbance from any real objects in order to produce the virtual hand on the screen. Figure 1 shows the setting for this workspace.

4 4 Figure 4: Workspace Setup 4.2 3D Gesture Inputs By using the Leap Motion Unity Core Asset, the physical interaction between the input and the objects in the virtual 3D environment now can be possible as the materials of the virtual hand have been provided. Next, the rigged hand script is executed from the Leap Motion Assets into the 3D hand model, making the virtual hand to successfully display on the screen.the gesture inputs that been used in this project were pinch, grab and release. The coding was implemented into the 3D virtual hand to invoke the 3D gesture inputs into the gesture tracking system. The pinch gesture posture reacts as the pointing device in the virtual world. As presented in Figure 2, the system detected the forefinger as the pointer marker and hover to the closest object. Figure 5: Pinch Gesture When the pinch gesture input was made, the ray cast mechanism detected the position of any closest object from the virtual hand model and later hovered it. The selected object now being ready to be grabbed by user s real hand using grab gesture input.

5 5 Figure 6: Grab Gesture Once the object being hovered, automatically the object is activated to be grabbed by the virtual hand using the thumb and forefinger as displayed in Figure 3. When the object was grabbed, a notification is sent to the system to give the feedback on screen. The feedback informs that the virtual hand is in the grab state. Then, in order to maintain the realism, the object should be aligned with the palm axis to perform rotation and translation. Figure 4: Release Gesture By releasing the thumb and forefinger as shown in Figure 4, the grabable object was release from the virtual 3D hand model. At this state, the system has been notified that the current state has changed from grabbed to release. 4.3 Developing 3D Pick Up Sticks Game The Pick Up Stick 3D game has been fully developed in Unity3D engine. The development of the concept and gameplay, game levels, GUI and sound were briefly explained in this section.

6 6 Figure 5: 3D Pick Up Sticks Gameplay GrabableObject.cs script is applied to the stick while the GrabbingHand.cs is applied to the 3D hand model in order to enable the physics of grab and release in the game. By finding the closest object that has rigid body attached at Figure 5 (a), the pinch gesture hovered the closest stick and made it available to be grabbed. While doing so at Figure 5 (b), the rigid body attached to the stick transformed its position to the player s hand. In this state, the player needs to careful grabbed out the stick from the other scattered sticks since it has collider to detect the collision of the objects. If the collider detects collision from the sticks, the game is ended. After the stick is successfully being grabbed and removed from the other sticks, the player can release the grabbed stick by releasing the grab gesture as shown as Figure 5 (c). Thus, making the combined rigid body of stick break from the player s hand. Lastly, the stick that been picked up was disappear from the scene upon being released signaling the player has gained points from the successful picked stick. Figure 6: Game Levels

7 7 In the level 1 displayed at Figure 6 (a), there are 5 sticks that scattered far from each other thus making it easier for the player to pick the stick without interference from other sticks. This level acts like a tutorial for the player to learn how to pick those sticks. Then, after the player finished picked all the sticks, the player proceeds to level 2 as shown in Figure 6 (b). In this level, the sticks number increased to 6 and it scattered near to each other signaling the difficulty of the game has increased. Finally at level 3 as displayed in Figure 6 (c), the sticks increased to 9 and the scattered gaps between the sticks become closer than the sticks at previous levels as the difficulty of the game reached at its peak. 5.0 Results This section will explain about the final Graphical User Interface of the game and the results of the Usability Testing that has been carried out. 5.1 Graphical User Interface (GUI) The Graphical User Interface (GUI) for this 3D Pick Up Sticks game is designed fully by using Adobe Illustrator CS6 and Adobe Photoshop CS6. Then, all the GUI is imported into the Unity3D using the GUI feature provided by the game engine. Some of the final GUI of the game can be found below. Figure 7: Main Menu Screen Every game has main menu. Main menu is important to redirect the user to the specific game tasks. In this case, by referring to the main menu at Figure 7, it has three events that are Play Game, Scoreboard and Setting. By using a mouse, the player can click at any buttons and the game will proceed accordingly to the specific tasks. Figure 8: Play Game Screen

8 8 When the Play Game mode had been chosen, the game will direct the player to the level 1 as shown in Figure 8. Level 1 is by far the simplest level in the game, only with a few sticks to be picked up. It is suitable for the novice player to learn the concept of the game. The top right shows the score of the player and the highest score will be listed on the scoreboard. Play Game mode has three level that are level 1, 2, and 3. The difficulties of the game increase gradually with the levels. Figure 9: Scoreboard Screen The scoreboard can be accessed from the main menu. The highest score achieved from the Play Game mode is listed here as shown in Figure 9. The scoreboard displayed the name and the score gained by the player. 5.2 Usability Testing Usability testing is carried out by given a questionnaire to the 10 tester. This stage intends to get user feedbacks and to evaluate the user interaction to the game s interface. Figure 10: Naturality using the Hand Gesture Interactions Figure 10 shows the bar chart results of question 1 and question 2 of the questionnaire. These two questions literally asked the player about the naturality using the Hand Gesture Interactions in the 3D Pick Up Sticks Game. From the bar chart, it can be

9 9 said that the testers agreed using the Hand Gesture Interactions in the game is not totally natural. This is because the testers were not familiar with the Leap Motion technology that required them to pick something in the virtual world, but in the reality world, they actually do not pick anything at all. Figure 11: Difficulties picking and rotation the sticks The result of the difficulties of picking and rotating the sticks is shown in Figure 11. Only 1 tester concludes the process of picking the 3D sticks was easy and the rest agreed it was hard to pick the 3D sticks using the Hand Gesture Interactions. They need some time to adjust how to get the correct angle in order to pick up the sticks. However, 5 testers agreed that the process of rotating the sticks was not too easy and not too hard, while 3 testers said that the process was easy. From this, it can be conclude that the process of rotating the sticks is easier than the process to pick up the sticks. Figure 12: Difficulties picking and rotation the sticks

10 10 The bar chart at Figure 12 displays the result of the game interface s efficiency. The efficiency of the game interface is in top notch as all testers agreed the task can perform correctly and quickly. This is because the buttons and icons in the game were simple yet interesting. 6.0 Conclusions The 3D Pick Up Sticks game using Leap Motion controller is an initiative to revive back the long forgotten traditional game while maintaining the feelings of realism playing the game. During the development of the project, the objectives mentioned in section 1 are successfully achieved to make sure a high quality game can be developed. The game can works as expected. However, the game still has flaws due to the limitation of Leap Motion s sensor. So in order to overcome the limitations of the game, some modifications and improvements needed to be done. The first thing that came in mind if given more time to deeply further the development of the game is changing the size of the sticks so that the leap motion can precisely detect the objects. For example, instead of sticks, the game can use sweets or gummi bear while maintaining the concept of the game to make full use of the Leap Motion sensory space. Besides, it also can attract kid to play this game. Next, the sticks can be increase to it maximum number that is 40 sticks to maintain the urge feeling of playing the real game. When the sticks getting increases, the difficulties to play the game will also been increases. Thus making it become more suitable to play multiplayer and online. Players can play the game together and if one of the players failed to pick up the stick, the turn will keep changing until the game is finished. The one who got the highest number of sticks will win the game. With this features, players can play pick up sticks from wherever and whenever they are with their friends across the globe. Last but not least, to make this game more interesting. The game can integrate with any virtual reality devices to let players experience the game in 3D virtual environment. With the help of virtual reality and hand gestures, the players can bring the game to life as it playing with the real sticks. References 1. Stone, B. (2010). The Children of Cyberspace:Old Fogies by Their 20s. The New York Times. 2. Google Play (2015). Pick-up sticks Android Apps on Google play. From: (Viewed: 26 April 2016). 3. The Editors of Encyclopaedia Britannica (2015). Pick Up Sticks. From (Viewed: 24 April 2016). 4. Alex Colgan (2015). 5 Medical and Assistive Technologies Being Transformed with Leap Motion. From: (Viewed: 24 April 2016). 5. Reiten, J.E. (2014). 3D modelling using Leap Motion. Norwegian University of Science and Technology.

11 11 6. Clements, D. H., Nastasi, B. K., Swaminathan, S. (1993). Young Children and Computers:Crossroads and Directions From Research. National Association for the Education of Young Children. 7. EDVENTURE playtime games (2014). Guide to Pick Up Sticks. From: (Viewed: 24 April 2016). 8. Fox, B. (2016). Redesigning our unity core assets for Orion. From: (Viewed: 26 April 2016). 9. Guna, J., Jakus, G., Pogačnik, M., Tomažič, S., and Sodnik, J. (2014). An Analysis of the Precision and Reliability of the Leap Motion Sensor and Its Suitability for Static and Dynamic Tracking. Faculty of Electrical Engineering, University of Ljubljana. 10. Haryth, A. (2016). Development of Block Building Game in a 3D Virtual Environment Using Hand Gesture Interaction. Faculty of Computing, Universiti Teknologi Malaysia. 11. Kaduson, H., Schaefer, C. (1997). 101 Favorite Play Therapy Techniques, Volume 1. Rowman & Littlefield Publishers, Inc. 12. Mitchell, K. (2014). TedCas strides toward the next-gen operating room - leap motion Blog. From: (Viewed: 24 April 2016). 13. Cardoso. J. (2013). Leap Motion considered great device for educational purposes by flow studio, hopes ecosystem emerges. From: (Viewed: 26 April 2016). 14. Leap Motion (2013). Solar walk - 3D solar system model. From: (Viewed: 24 April 2016). 15. Karamian, V. (2015). Unity3D leap motion integration. From: (Viewed: 25 April 2016). 16.Lee, M. (2015). New stroke therapy uses motion sensor video game to help rehabilitation. From: (Viewed: 24 April 2016). 17.Stolovitch, H. D., Keeps, E. J. (2006). Handbook of Human Performance Technology. John Wiley & Sons, Inc. 18.Smith, C. (2014). Tracking hand tremors with leap motion. From: April 2016). 19.Sobre, J.G. (2014). Leap motion para Autodesk maya From: (Viewed: 24 April 2016). 20.Triplett, D. (2015). Pick-up-sticks for business and life. From: (Viewed: 24 April 2016).