PhysX-based Framework for Developing Games with Haptic Feedback

Transcription

1 PhysX-based Framework for Developing Games with Haptic Feedback R.P.C. Janaka Rajapakse* Yoshimasa Tokuyama** and Kouichi Konno*** Tainan National University of the Arts*, Tokyo Polytechnic University**, and Iwate University*** and Abstract The main goal of computer game is to immerse a user in a virtual environment. In traditional games, the user immersion is performed through the sense of visuals and sounds with interactions through 2D input devices like mouse and keyboards. Haptic technology can simulate tactile and kinesthetic sensations in virtual environments. Recently, there are a variety of games which take advantage of the haptic effects offered by some devices such as joysticks, gamepads and haptic devices. The main criticism of haptic-driven game development is that the real-time interactions must allow graphical updates of 30 Hz and a much higher rate of 1 khz for force feedback. The rapid development of the physics engines in recent years has enabled real time simulation of multi-physics effects for 3D games. The major challenge of using such physics engines for realistic haptic rendering is the computation of physically simulated force at interactive haptic rates. This paper presents a framework for developing haptic-enabled 3D blocks arranging games like virtual Jenga and Tsumiki based on NVIDIA s PhysX application programming interface (API). The proposed framework introduces a collision handling method to calculate feedback forces by using the collision geometry data obtained from PhysX and the haptic rendering device to provide force feedback. 1. Introduction Immersive is a widely used term in interactive applications such as virtual reality, computer games, and human-computer interaction. The main objective of immersive systems is to artificially create all stimuli that would be experienced in that environment in reality. Immersing someone in a virtual world can be done by many different ways. Touch is a powerful sense for humans; hence, haptic interaction with vision offers users new immersive experiences with applications in entertainment and hyper media [1, 2, 3]. Haptic technology and simulating tactile feedback for virtual reality applications has been widely studied [4, 5, 6]. Recent years have seen rapid growth of popularity of computer gaming. According to the annual report of Entertainment Software Association (ESA), this industry has been totaled over eleven billion dollars in US sales alone in year 2008 [7]. Meanwhile, the rapid growth of the computational power of central processing units (CPUs) and graphic processing units (GPUs) in recent years has begun to expand multimodal interaction for game consoles and home computers. With the advent of first inexpensive haptic devices on the consumer market, game industry had focused on introducing tactile feedback to the players [8]. In most current games, haptic sensation is performed through the sense of vibrotactile feedback by using devices such as two-degree-of-freedom (2-DOF) joysticks, gamepads, drumsticks, steering wheels, airplane flight controllers, etc. A way to increase the user immersion into the games is to make use of more sophisticated haptic device. One of the major challenges in haptic rendering for games is the computation of continuous forces at haptic rates. To address this issue, a haptic-driven game must refreshes the screen Hz to give the user feeling of continuous movement while haptic device is permitting a much higher rate of 1 khz for force feedback. A solution to this barrier can also be utilized by integrating with physically-based simulation of rigid body dynamics, which has become increasingly popular in games over the past few years. Recently several physics engines have been developed which offer multi-physics simulations for 3D games [9-13]. Even though such physics engines and libraries provide unique opportunities for the development of games, less integration offers to their use in the development of interactive haptic games. The main objective of this work is to investigate the potential of using NVIDIA s-physx physics library [9] to calculate feedback forces for haptic-enabled games and virtual reality applications. In this paper,

2 we present a framework for developing haptic-driven 3D blocks arranging games such as like Jenga and Tsumiki (See Figure 1 and Figure 2). In 3D blocks arranging games, it is very important to create realistic interaction between the blocks while player is feeling the haptic sense of them. Simulating contact and collision for a large number of blocks is a big challenge. Figure 1: Tsumiki - blocks arranging games The proposed framework introduces a collision handling method to calculate feedback forces by using the collision geometry data obtained from PhysX and the haptic rendering device to provide force feedback. Figure 2: Jenga - blocks arranging games The rest of this paper is organized as follows. Section 2 reviews background literature on physics engines and haptic interaction in games. In Section 3, we introduce procedures of proposed PhysX-based framework for the development of 3D games with force feedback. This Section also shows overview of the system and implementation details. Results are presented in Section 4. Finally, we conclude in Section 5, and discuss possible future work. 2. Background There have been numerous research work done on haptics and real-time physical simulations in computer graphics history. The purpose of this short review is to examine available physics engines, haptic rendering and haptic interaction in games. 2.1 Physics Engines Extensive research has been concerned on the simulation of dynamic rigid bodies in computer graphics. In the early days, quite simple approaches were used to model physical behavior such as mass-spring networks or particle systems. Modern physics engines can often handle much more, such as cloth simulation, fluid dynamics, deformable objects, etc. We consider here a list of currently available physics engines that handle more or less complete multibody dynamics. Most of them are designed for developing games and entertainment applications. PhysX is a dynamic physics engine solution initially launched by Ageia software, then acquired by NVidia [9]. PhysX engine leverage the parallel computational power of hardware, including modern GPUs and physics processing units (PPUs) to accelerate computation time. It proposes complete dynamic constraint resolution, including collision detection and physically based simulation of rigid bodies, cloth, soft-bodies, fluids and destructions. Havok Physics is a real-time collision detection and physical simulation solution [10]. Havok Physics has been widely awarded for its robustness, scalability and team support. It includes complete detection system and optimized dynamics and constraints solver. Havok Physics is actually a component of the Havok's suite, which currently includes Havok Animation, Behavior, Cloth and Destruction. Bullet Physics is an open source dynamic engine which provides collision detection, and soft and rigid body dynamics [11]. It is an open source competitor for Havok Physics or PhysX engine. Open Dynamic Engine (ODE) is an open source dynamic engine, handling articulated rigid body dynamics [12]. It is a constrain-based physics engine that uses Euler integrator and fixed time stepping. Newton is a free physics engine that is made for several platforms, Window, Mac, and Linux [13]. It is an open source dynamic engine for rigid bodies. There are not much works done testing and comparing different physics engines. Physxinfo.com [14] published a web-based article comparing gathered statistics of three different physics engines, PhysX, Havok physics and ODE. This article has tried to summarize what PhysX SDK has archived in game industry. Their statistical evaluation did not focused on terms such as performance, features, quality, etc. but considered released game titles in recent years. Figure 3 shows the numbers of released game in time period from

3 Figure 3 shows detailed statics on number of released games per quarter of calendar years more detailed surveys on haptic rendering. However, it broadly consists of two processes: collision detection and collision response. In the particular case of a force-feedback device such as the PHANTOM [29], being an input and output device, if the operator touches a solid wall from the haptic model the collision-response mechanism transmits the right force to the device to mimic a solid surface. Thus, when the operator tries to push through the solid wall the device transmits a greater force. A useful survey on collision detection is by Lin et al. [18], it provides a good overview of the main collision detection algorithms. Weller et al. [19] presented a hierarchical data structure which makes the algorithm suitable for haptic rendering of very complex objects with several thousands of polygons. Figure 3: Number of released titles (source from As shown in Figure 3 and Figure 4, there is an increasing adoption of PhysX by game industry, because of the integrated hardware acceleration and the collection of help functions as well as the great established communities for developer support. Figure 4: Number of released titles (source from Without providing a quantitative evaluation of available physics engines, it is very difficult for a developer to select an appropriate physics engine for their developments. Boeing et al. [15] published an article quantitatively comparing publicly available physics engines for simulation systems and game development. According to their evaluation, no one engine performed best at all task, and almost every test was performed best by different engine. To our knowledge, there are no publicly available physics engines that provide direct implementations for haptic-enabled collision calculations. Therefore, this work is to focuses on the potential of using NVIDIA s-physx physics library to calculate feedback forces for haptic-enabled games and virtual reality applications. 2.3 Haptic interaction in Games In the literature, realistic haptic interaction using in 3D games are not very common. Typically haptics are used in PC and console games through the sense of vibrotactile feedback by using devices such as two degree of freedom (2-DOF) joysticks, gamepads, drumsticks, steering wheels, airplane flight controllers, etc. [8]. Problem with the typical vibrotactile feedback of traditional game controllers is that it is too ambiguous: it doesn t give specific information to the player of what has happened. To significantly improve the usefulness of tactile feedback, it needs innovative techniques. Jiang et al [20] has found that well-designed haptic feedback reduce error rates and improve performance. In recent years many interesting innovations have expanded the usage of novel sensors and multi-touch displays in console and mobile games. The Nintendo Wii [24] has introduced an entirely new approach to gaming. When we compare games that are available on Wii with those for the Xbox360 and the PlayStation 3(PS3) [25, 26], it is clear that Wii s graphics are not on par with the other consoles. However, its motion-sensing game controller has made it a success story worldwide [21], outselling both Microsoft s and Sony s consoles. While some novel technologies are enhancing performance and environment in the field of console gaming, there has been little progress in haptic-enabled PC gaming. Recently Novint launched a low-cost haptic device named Novint-Falcon for the gaming market [27] (See Figure 5). 2.2 Haptic Rendering Haptic rendering is the process of computing the force required by contacts with virtual objects based on measurements of the state of the device. This state may be modified by the operator s applied position, force, torque, and/ or environment dynamics [1, 2, 3, 4, 5, 6]. In the book chapters, Salisbary et al. [16] and Lin et al. [17] have reviewed Figure 5: Photo of Novint Falcon [27]

4 Morris et al. [22] introduced Haptic Battle Pong, a competitive networked game that makes extensive use of 3-DOF force-feedback and 6-DOF input. This is a pong clone with haptic control through the Sensable PHANTOM device. This is among the first games to fully utilize the capabilities of high-fidelity haptic devices, and explored the applicability of high-degree of freedom haptics to games. Andrews et al. [23] presented a game which named HaptiCast acts as an experimental framework for assessing haptic effects in 3D games. In this game, players have to assume the role of a wizard with an arsenal of haptically-enabled wands which they may use to interact with the game. Our framework is focused on the development of haptic-enabled 3D games with the NVIDIA PhysX engine in order to provide more effective PhysX accelerated collision handing method for 3D blocks arranging games. 3. System Overview Figure 6 depicts the outline of the system diagram. A PHANTOM Omni desktop device by SensAble Technologies [29] is employed as a haptic device for 6-DOF input and 3-DOF feedback of force. The system consists of two sub modules for haptic rendering and physics simulations. The haptic rendering module of the system was developed with the OpenHaptics toolkit to read current positions of haptic device and rendering feedback force. The physics simulation module uses new position of the object and makes physics simulation of the scene, and then passes detected collision geometric data to control the haptic device via haptic rendering module. Figure 7: Mass-spring-damper system excited by force F. External force F can be computed by solving following differential equation Force on the Object To control positions of the body, basic linear mass-spring-damper system is considered, oscillatory force and damping force can be computed as follows. Damping force: Total force on the body is, Since The natural frequency (undamped) of the system and damping ratio can be defined: The damping coefficient,, damping ratio determines the behavior of the system. The damping ratio is adjusted according to the simulation speed and stability. Figure 6: System Overview. 3.1 Force Simulation The most physics friendly way of interacting with a body is to apply external force. In classical mechanics, most interactions between bodies are typically solved by Newton s laws. In order to compute physically based force calculations on 3D blocks, we based on mass-spring-damper system [31] as show in Figure 7. It shows an ideal mass-spring-damper system with mass (m), spring constant (k) and viscous damper of damping coefficient (c). : means overdamped, the system returns to equilibrium without oscillating. : means system has critically damped, it returns equilibrium as quick as possible without oscillating. : the system oscillates with the amplitude gradually decreasing to zero, this state called underdamped. : means undamped and system oscillates at its natural resonant frequency. Assuming that the system has critically damped, and using the Equation 6, contact position of the object and components of the contact force can be calculated. The proxy mass and the spring stiffness are initializing manually, therefore force on the object will not be too large and not damage the haptic device.

5 3.1.2 Torques on the Object In the rigid body simulation, the torque (τ ) usually computed as follows. We considered dynamics of torsion spring [32] to control yaw, pitch, and roll torques on the object. τ θ (7) Where τ is the torque, θ is the angle of twist from its equilibrium position, and is called torsion coefficient or torsion elastic modulus. If torsion spring is not twisted than its elastic limit, dynamic of torsion follows angular form of Hooke s law. Figure 8 illustrates schematic diagram of three torque components on a 3D block. HL_EFFECT_START HL_EFFECT_STOP HL_EFFECT_COMPUTE_FORCE void HLCALLBACK computeforcecallback(hddouble force[6], HLcache *cache, void *userdata) In PhysX developments, entities are created as scene objects (NxScene). Scene objects are initialized by the NxPhysicsSDK. The basic PhysX objects called actors (NxActor), actors are in static and dynamic modes. In our developed games, actors are primarily design as primitive rigid body elements. These primitives represent the physical space the actor occupies in the simulation, and are the core of collision. PhysX NxD6Joint class is used to define custom joints in the physics computing. This 6-DOF joint provides motor drive on all axes independently and also allows soft limits. Force and torque on 3D blocks in the games can be computed using PhysX NxD6Joint. Multiplayer interaction in our proposed haptic-driven games is possible between the two players by enabling dual haptic interaction. Figure 8: The gimbal on 3D block that contribute the PHANTOM s 6-DOF yaw, pitch, and roll torques, respectively. During the torque simulations, damping ratio should be less than one and greater than zero. It means mass-spring-damper system is under damped when torques is applied on the joint of the object. Similar to the force calculation on the object, spring stiffness, and damping ratio are assign manually so that the torque would not damage haptic device. 3.2 Implementation This framework is implemented in Visual Studio 2005 development environment, and used C++, OpenGL API for graphic rendering. The proposed framework used NVIDIA s PhysX-API library to calculate collision geometric data and physical simulations of the rigid bodies. To control the haptic device, Sensable s OpenHaptic-3.0 toolkit is used in the implementations [30]. The proposed framework developed haptic-enabled 3D block arranging games like virtual Jenga and Tsumiki, basically consisting primitive geometric shapes. In our developments, we used HLAPI in conjunction with QuickHaptics micro API to take the advantage of handling the computations of haptic rendering based on geometric primitives, transforms, and material properties. The Shape Class in the QuickHaptic API defines all of the geometry primitives that can include in the world space. HLAPI is used to implement PhysX accelerated custom force effect using a callback function interface. There are three main callback functions that need to be registered for proper execution of a custom force effect. 4. Results Our framework was performed on a standard PC equipped with a 3.33 GHz Intel Xeon CPU and a NVIDIA Quadro FX 4600 graphics board with a 768MB video memory. This framework developed two haptic-driven games which refresh the screen Hz to give the user feeling of continuous movement of virtual scene while haptic device is permitting a much higher rate of 1 khz for force feedback. In both games, user can interact with 6-DOF inputs and can immerse with virtual scene by 3-DOF haptic feedback. The first game is named simple Tsumiki, a 3D blocks arranging game with realistic haptic feedback. Figure 9 shows two screen shots captured from haptic-enabled 3D Tsumiki game. Left image of the Figure 9 was captured when player was arranging the 3D blocks, and right image was taken after arranging. This simple Tsumiki game can obtain 60 FPS in graphic rendering with realistic haptic interaction. The second game is called 3D Jenga, haptic-enabled 3D blocks arranging game consisting 65 small blocks. Figure 10 shows few captured screen shots while player was arranging 3D blocks. The 3D Jenga game can obtain graphics rendering rates in between 30-60Hz while enabling realistic haptic feedback. 5. Conclusion In this paper we have reviewed some of the recent developments in the field of enhancing gaming through the use of haptic and physics engines.

6 (a) (b) Figure 9: Two screen shots captured from haptic-enabled 3D blocks arranging game called Tsumiki, (a) while playing and (b) after arranging Figure 10: The screen shots captured from haptic-enabled 3D blocks arranging game, named Jenga We have successfully introduced a framework for the development of haptic-enabled 3D games. We have examined the potential of using NVIDIA s-physx physics library to calculate feedback forces for haptic-enabled games and virtual reality applications. The techniques used in this framework have potential for reducing development time and enabling haptic interaction that immerse virtual reality applications. Based on this pilot framework, the development of more sophisticated blocks-arranging games will be focused. Additionally, we are extending this Tsumiki game with more complex 3D models by using PhysX s NxTriangleMeshShape class and QuickHaptics s TriMesh class. Furthermore, hardware accelerated data-parallel implementation of the force computation should result in further speed-ups. Acknowledgement The authors would like to acknowledge Eri Takahashi and Huang Ming-Min for their help with creating video and image files. Authors also wish to thank Venkat Gourishankar at Sensable Forum for providing valuable comments and guidance on some implementation issues.

7 Reference [1] S. J. Biggs and M. A. Srinivasan, Haptic Interfaces, Handbook of Virtual Environments: Design, Implementation, and Applications, (Human Factors and Ergonomics), K. Stanney, Ed. Lawrence Erlbaum Associates, Ch. 5, pp , [2] H. Iwata, Artificial reality with force-feedback: Development of desktop virtual space with compact master manipulator, in ACM SIGGRAPH, vol. 24, no. 4. ACM Press, pp , [3] V. Hayward, O. R. Astley, M. Cruz-Hernandez, D. Grant, and G. R. De-La-Torre, Haptic interfaces and devices, Sensor Review, Vol. 24, No. 1, pp , [4] C. R. Wagner and R. D. Howe, Mechanisms of performance enhancement with force feedback, Proc. IEEE Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, pp , [5] M. Benali-Khoudja, M. Hafez, J.-M. Alexandre, and A. Kheddar, Tactile interfaces: a state-of-the-art survey, in ISR 04, Paris, pp , [6] R. J. Stone, Haptic feedback: A potted history, from telepresence to virtual reality, in Workshop on Haptic Human-Computer Interaction, ser. LNCS, S. Brewster and R. Murray-Smith, Eds., vol. 2058, pp. 1-7, [7] Video game industry, Wikipedia online article, 15 th May [8] D. Chang, Haptics: Gaming s New Sensation, Computer, Vol. 35, No. 8, pp , [9] PhysX, (on 15 th May 2011). [10] Havok Physics, (on 15 th May 2011). [11] Bullet Physics, (on 15 th May 2011). [12] Open Dynamic Engine (ODE), (on 15 th May 2011). [13] Newton, (on [14] Popular Physics Engines Comparison, Online Article, published by PhysXinfo.com [15] A. Boeing and T. Braunl, Evaluation of Real-time Physics Simulation Systems, Proc. GRAPHITE 2007, pp , [16] K. Salisbury, F. Cont and F. Barbagli, Haptic Rendering: Introductory Concepts, IEEE Computer Graphics and Applications, Vol. 24, No. 2, pp , [17] M. C. Lin and M. A. Otaduy, Eds., Haptic Rendering- Foundations, Algorithms, and Applications. AK Peters, [18] M. Lin and S. Gottschalk, Collision detection between geometric models: a survey, in IMA Conference on Mathematics of Surfaces, pp , [19] R. Weller and G. Zachmann, A unified approach for physically-based simulations and haptic rendering, Proc ACM. SIGGRAPH Symposium on Video Games, ACM press, pp , [20] L. Jiang, R. Girotra, M.R. Cutkosky and C. Ullrich, "Reducing error rates with low cost haptic feedback in virtual reality-based training applications", in Proc worldhaptics, pp , [21] P. Ramsamy, A. Haffegee, R. Jamieson, and V. Alexandrov, Using Haptics to Improve Immersion in Virtual Environments. Lecture Note in Computer Science, Volume 3992/2006, pp , [22] D. Morris, J. Neel, and K. Salisbury, Haptic Battle Pong: High-Degree-of-Freedom Haptics in a Multiplayer Gaming Environment. Experimental Gameplay Workshop, GDC [23] S. Andrews, J. Mora, J. Lang, and W. S. Lee, HaptiCast: A Physically-Based 3D Game with Haptic Feedback, in the Proceedings of the FuturePlay [24] Nintendo Wii., (on [25] Microsoft Xbox360, (on [26] Sony, PlayStations 3, (on [27] Novint Falcon, Novint Technologies. (on [28] Delta. 3, Haptic Device and Reachine Academic Pack API, (on [29] PHANTOM haptic devices by Sensable Technologies, (on [30] OpenHaptics Toolkit, [31] Mass-spring-damper, (on [32] Torsion spring, (on