Rendering detailed haptic textures

Transcription

1 Workshop On Virtual Reality Interaction and Physical Simulation (2005) F. Ganovelli and C. Mendoza (Editors) Rendering detailed haptic textures Submission id: 45 Abstract Rendering haptic textures seamlessly out of triangle meshes just by using geometry requires heavy work and does not allow high sampling rates for detailed models. Better approaches simulate surface texture without increasing much the complexity or processing cost, at the expense of fidelity of perception. We propose a prism-based method for rendering local height field maps out of an underlying triangle mesh, which relies in a space subdivision representation based in octrees for collision detection, and locally rendering individual surface detail by modulating force response using local height fields. We compare our method against a force mapping implementation for rendering the same models and textures using normal maps. The proposed technique allows for real time perception of 3D surface detail at various resolutions, allowing the user to perceive the best rendering alternative for a given model. Categories and Subject Descriptors (according to ACM CCS): I.3.3 [Computer Graphics]: Haptic Rendering physical object and surface modeling, volume processing, and the visualization of novel data structures able to encode shape and material properties. From single one-point based, single person operation to multipoint, multihand, and multiperson interaction scenarios, an enticingly rich interactivity is within reach of many computer graphics applications. 1. Introduction Figure 1: Height field texture rendering Haptic systems provides unique and bidirectional communication between humans and virtual environments in a manner much closer to usual physical environment manipulation. Haptic interfaces enables direct interaction with computergenerated objects, and when coupled with an intuitive visual display of complex data raise applications to new levels; these applications include molecular docking, nanomaterial manipulation, surgical training, virtual prototyping, machine assembly and digital sculpting. Haptics drives the development of new algorithms for One of the interesting applications on haptic perception is to be able to feel, with a haptic device, variations in texture, roughness or details of the surface being contacted. There has been some work done in this direction, but none compare the results of applying different techniques to the same model. The general idea, as in visualization, would be the simulation of roughness and other surface features without having to increase the geometric density of the model. In this paper, we compare three possible solutions, analizing their advantages and disadvantages, being the last one our solution for haptic perception. The contribution is a specific model for simulating haptic perception of surface details, by using a per-face height field as a texture to be mapped onto the objects geometry. We achieve a good correspondence between surface detail visualization and haptic perception (figure 1). The paper is organized as follows: in section 2 we present related work recently done in haptic rendering. Section 3 explains the solutions we have compared, with one of them being the approach we present in this paper. In section 4 rec The Eurographics Association 2005.

2 sults are presented and discussed, and finally the conclusions and future work. 2. Related work The term haptic rendering is applied to the generation and rendering of haptic virtual objects [LS04]. Although there has been some work in pseudo haptics by simulating surface properties using common computer mice [LBE04], it is more common the use of a specialized haptic interface device producing a force-feedback response. Users can manually navigate, explore and feel the shape and surface details of virtual objects in the growing field of computer haptics [BS02]. As sampling rates for sensing devices standardizing in the 1000 Hz range, as in the Phantom or HapticMaster device [FCS02], efficient haptic-interaction techniques may go beyond the simple detection of geometric primitives, towards real-time rendering of arbitrary surfaces of irregular detail or convexity, conveying surface geometry and material properties, without forgetting its role of user-interaction device for high level event input, recognition of haptic icons and general haptic user interfaces or HUIs [ME03]. The most simple haptic model uses simple surfaces, a point-based device and a force-collision detection algorithm [OL05]. A haptic cursor representing a force-feedback device is placed into a 3D environment, and a high speed event loop checks whether it collides with a surface of an object, after which it produces a repulsing force of varying direction and magnitude, which physically acts upon the force exerted by the user in the haptic device, correcting any penetration ruled out by the object s geometry [MH04]. Using a ray-based rendering algorithm with the same setup allows perceiving torque and force-torque feedback mechanisms [MS05], while using a third object as a probe allows also texture diferentiation and shape perception [OL03]. There is an ongoing effort for developing libraries and frameworks to incorporate haptic capabilities to applications, such as OpenHaptics(TM) [IHZ05] with separate modules for haptic device and user interaction. Other approach is connecting a haptic rendering library, its graphical twin, and a collision detection module in a generic framework [LBDM04]. These efforts choose among several alternatives for modeling and rendering surfaces. Johnson [JW04] uses a pure geometric modeling approach for haptically render arbitrary polygons using neighborhood proximities to reduce computational load. Some haptic techniques and approaches can be found in [BS02], with a treatment of force mapping that simulates detail at higher resolutions by perturbing surface normals, therefore modifying the surface texture perceived by the haptic probe. Potter [PJC04] provides a simple model to perceive the haptic texture of large heightfield objects. Multiresolution techniques [AB01] are also used to generate smooth geometry from large heightfield objects, allowing a better simulation of terrain models. Based on all these works we can summarize a taxonomy for haptic detail rendering, which determines the particular algorithm to be used. Geometry, rendering the surface as detailed polygonal meshes, NURBS, or point clouds, and detecting collisions against the surfaces. Surface Relief Detail, in which a haptic texture is sampled in lieu of the actual surface. On its own, haptic textures may be based on bump maps, force fields, or height fields. These approaches may also allow managing LOD in the visual and haptic resolutions, and having a repository of assorted relief textures. In the next section, we present a comparison among three different techniques for the surface relief detail haptic perception. 3. Models for haptic perception of surface details The simulation of surface details in very complex models has not been a problem from the visualization point of view since the late 70 s. Algorithms such as the use of colored textures or bump-mapping are well known in the literature [Bli78]. In the case of haptic perception, as we have seen in section 2, any simulation algorithm should be efficient enough to achieve the high frequency updates (1000 Hz) required by the force feedback devices. Our objective is to find an algorithm to allow haptic perception of surface details in very complex scenes represented by triangle meshes, and to compare it to other known solutions. The experimental setup is based on an algorithm for the generation of a detailed geometry, a force mapping algorithm based on normal maps, and the solution proposed which incorporates height field mapping (see figure 2) Generation of geometry A direct algorithm simulates surface details by generating the local geometry corresponding to these details. This geometry can be calculated as a preprocess for the whole model, or locally for each triangle of the mesh at run time, by deciding which triangles of the mesh are inside the haptic influence area. A collision detection algorithm then tracks the haptic probe (or god-object), which is represented in the screen by a proxy. When moving freely in empty space, its coordinates coincide. When the god-object hits a triangle, the proxy clamps to the collision point, and a force is exerted in the haptic device to push the god-object to the nearest point in the hit triangle, with the proxy following suit along the surface.

3 (a) Geometry (b) Force mapping (c) Height fields Figure 2: Different approaches to simulate surface details in haptic perception In the afore-mentioned cases, the cost in time and/or space of modeling detailed surface using just geometry is not affordable given the frequency updates requirement. Having many small triangles, the haptic probe may well be at another point in the surface (a far-off triangle) after detecting a collision, and the resulting force interplay can cause unstable behavior, adding spurious force feedback, inducing vibration or simply wrong sampling (haptic aliasing) Force mapping One of the algorithms used in visualization to simulate surface details is the bump-mapping algorithm. It consists of a texture map where each value corresponds to a normal vector direction (RGB corresponds to XYZ coordinates respectively), which can be different for each point in the map. A discretization of the triangle s surface is achieved by mapping texture points to corresponding points within the mesh triangle, having a different normal vector at each point. The visualization bump-mapping algorithm uses these normal vectors at the discrete points to calculate illumination values inside the triangle, simulating the surface details. The haptic perception bump-mapping algorithm (also called force mapping) [BS02] is based on the same principle. It uses the normal vector at discretized surface points to calculate the force direction and magnitude to be applied to the haptic device when it collides with the triangle (see figure 3). The algorithm used in this case can be described as follows: for each new haptic device position do detect collision with the mesh if collision exists then // we have the collision point in the triangle calculate force direction from the map at that point apply this force to the haptic device endif endfor By using this haptic perception algorithm and implementing the bump-mapping visualization as a GPU shader, we achieve a correct perception of surface roughness when the distortion is not very high. This is because the collision is al- Figure 3: In 2D: correspondence between simulated surface (up) and normal vectors (down). ways detected against the triangle of the mesh, so a perception of displacement upward or downward from the triangle surface is not possible (more detailed results in section 4) Height field haptic rendering Height field rendering has been used primarily as a technique for imaging large datasets out of GIS data. These are usually huge files stored as RGBα images, with the RGB providing texture information and the α channel providing the vertical height. A grid is used to scan the height field, interpolating and filtering values as needed, and creating a corresponding geometry. For haptic rendering in this sort of applications (terrain models), the proxy s calculated position must correspond accurately to that of the tracked god-object s, because the magnitude of the height field map is usually very big and small differences may cause wrong perception. These models generally render a whole terrain as a gigantic height field map [PJC04]. The requirements can change slightly when the objective is not a GIS application. In our case, our interest is using height field mapping applied to haptically render a triangle mesh model, in which different height field textures can be applied to different triangles. Another requirement in this

4 sort of applications is that the triangles are not as big as the whole model, and actually in many cases they are small enough to be represented in a small area of the screen. This implies that the accuracy required for computing the proxy s position as explained above is not so strict in this case, since we want to simulate small surface details in a single triangle of the mesh. On the contrary, it becomes much more important to be able to determine, at a low cost, which triangle is being collided by the haptic probe, so the haptic render algorithm can apply the appropriate height field mapping. In order to fulfil this requirements our algorithm combines different techniques to make this haptic rendering as efficient as possible: the triangle mesh is represented by an octree space decomposition, which allows a very quick detection of the area where the haptic probe is located, discarding early in the process a very high amount of geometry of the model which does not need to be considered in the rest of the algorithm; the collision with a triangle of the mesh is detected against a prism created between the triangle and a copy of the triangle displaced over a certain maximum distance for the height field map values (see figure 4(a)). All triangles created to define this prism share the same identifier with the original surface triangle of the mesh, so the computation of the right triangle is inmediate; a potential collision against any side of the prism triggers the height field haptic rendering of a small grid cell just lying in a perpendicular triangle under the probe, shadowing the probe s movement. If at any time the probe descends below the signaled height for that cell, a repealing force is applied to the haptic probe along the surface normal at the proyected point, proportional to the height difference (or penetration). This forces the god-object onto the surface at that point, at which the force ceases to be (see figure 4(b); all the height field maps are forced to distance zero on the edges of the mapped triangle, avoiding discontinuities on these edges that can cause instability in the haptic device; The algorithm used for this approach works as follows (see figure 4): for each new haptic device position do detect the octree node detect collision with a triangle prism if collision exists then // the god-object position is inside the triangle prism project the god-object position to the triangle of the mesh obtain the height of this point from the height field if (distance(god-object, triangle) < height field) then // we had a collision calculate an outward force proportional to the relative height differences apply this force to the haptic device endif endif endfor (a) Normal map (ovals) (d) Height field (ovals) 4. Results (b) Normal map (cross) (e) Height field (cross) Figure 5: Normals and Heights maps (c) Normal map (warts) (f) Height field (warts) In order to compare the quality of haptic perception using both force mapping and heigt fields, we decided to set up the environment experimental conditions as close as posible for each run. This means using the same base models for both runs, and an exact correspondence between the relief map used as a height field and the normals map used for force mapping. First, a sphere, built recursively at several resolutions out of a perfect icosahedron was chosen as the base model. For the sake of enhancing the features detailed in the article, it has been set to the simplest possible resolution for the figures appearing in this article, to better appreciate visually the rendered surface texture. Secondly, we generated synthetic height field maps, and used it to compute a corresponding normal map, using a plugin for the open source GIMP application. In this way we obtained corresponding pairs of height field and normal maps representing the same 3D geometry, as shown on figure 5. In the end we settled for the three textures shown here, since each allows to perceive different surface characteristics. Again, for the sake of visualization, the relief detail has been greatly exagerated. The textures chosen are an embossed cross with only flat and vertical surfaces, gently sloping circles with a soft gradient, and a texture with alternating polished and variable bumpy areas. Additionally, taking advantage of the graphics card hardware, we implemented the shading part corresponding to our bump mapping as a GPU shader, just to allow a better haptic sampling rate.

5 m h r f Submission id: 45 / Rendering detailed haptic textures w i = v i + n * m h w 3 v 3 c t w 1 c t w 2 H F u v p r o x y p e n e t r a t i o n g o d o b j e c t p o s i t i o n v 1 v 2 c t (a) Triangle box collision area (b) Collision point computation Figure 4: Height field collision mapping We show on figure 6, different renderings using force mapping and height fields. It is evident that bump mapping makes a smoother visualization, and it is much faster, even without the shader. However, the perceived sensation at the end of the probe does not quite convey a sense on the surface, and although gives a good approximation for modeling the apparent roughness of the material, it is a turbulent ride when using very irregular normal maps, such as those arising from terrain modeling, and other textures that need a sense of going up or down, even at smal steps. On the other hand, rendering height fields, as in the approach presented, gives an accurate sense of the surface characteristics, allowing 6 DOF movement while keeping the computing cost relatively low, as long as the underlying textures below the triangles are of a reasonable size. Individual local height fields between 128 and 256 pixels per side work reasonably well. Anything lower feels coarse and higher texture densities may be too much, either affecting haptic sampling rates, rendering framerates or both. The equipment used is a HapticMASTER from FCS, having a built-in wide haptic 3D space, able to exert forces from a delicate 0.01 N upto a very heavy 250 N, with a wide 3D perception field, equivalent to a wedge of 40 cm x 36 cm x 1 radian. The PC is a 2GHz Pentium IV with an ATI Radeon 9700 graphics card. By modulating the underlying texture by subsampling or gaussian smoothing of texture patches, the same surface can be rendered as grainy detail, a softer sloping surface or blocky one, altering the perceived qualitys of the same trangle mesh. There can be several texture maps active and rendered at any time, one for each base triangle. 5. Conclusions We have developed a practical method for rendering local haptic textures in triangle meshes, which allows an user to perceive exact surface details at several resolutions. This extends the use of height field haptics beyond the usual field of gigantic terrain textures and allows perceiving resolution of surface detail. This approach can be used for mapping relief textures in triangular meshes and haptically render them in real time. We are extending further this research by exploring a superposition of approaches: a force map rendering applyied to a height field on top of a triangle mesh, applying a dual relief texture. We also plan to work on fast algorithm for finding the exact height field collision using a local search instead of the normal projection method explained here, together with a faster, GPU-based method of rendering them in hardware, as to increase sampling rates and allow higher texture resolutions. This will allow us to extend the algorithm to render haptic detail of geometric models undergoing deformations in real time. References [AB01] [Bli78] [BS02] [FCS02] [IHZ05] [JW04] ASGHAR M. W., BARNER K. E.: Nonlinear Multiresolution Techniques with Applications to Scientific Visualization in a Haptic Environment. IEEE Transactions on Visualization and Computer Graphics 7, 1 (January/March 2001), BLINN J. F.: Simulation of wrinkled surfaces. In Proceedings of SIGGRAPH 78 (1978), ACM. BASDOGAN C., SRINIVASAN M. A.: Haptic rendering in virtual environments. In Handbook of Virtual Environments, Stanney K., (Ed.). London, Lawrence Earlbaum, Inc., 2002, ch. 6, pp , FCS CONTROL SYSTEMS: HapticMASTER Installation Manual. The Netherlands, December ITKOWITZ B., HANDLEY J., ZHU W.: The OpenHaptics? Toolkit: A Library for Adding 3D Touch? Navigation and Haptics to Graphics Applications. In Proceedings of the First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (2005). JOHNSON D. E., WILLEMSEN P.: Accelerated Haptic Rendering of Polygonal Models through Local Descent. In Proceedings of the 12th International Sympoc The Eurographics Association 2005.

6 (a) Cross (using force maps) (b) Cross (using height fields) (c) Ovals (using force maps) (d) Ovals (using height fields) (e) Warts (using force maps) Figure 6: Height field collision mapping (f) Warts (using height fields) sium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (HAPTICS?04) (2004). [LBDM04] LUCIANO C., BANERJEE P., DEFANTI T., MEHRO- TRA S.: Realistic Cross-Platform Haptic Applications Using Freely-Available Libraries. In Proceedings of the 12th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (HAPTICS?04) (2004). [LBE04] LÉCUYER A., BURKHARDT J.-M., ETIENNE L.: Feeling Bumps and Holes without a Haptic Interface: the Perception of Pseudo-Haptic Textures. In Proceedings of CHI 2004 (2004).

7 [LS04] [ME03] [MH04] LIN M., SALISBURY K.: Haptic rendering beyond visual computing. IEEE Computer Graphics and Applications (March/April 2004), MACLEAN K., ENRIQUEZ M.: Perceptual design of haptic icons. In Proceedings of the Eurohaptics 2003 (Dublin, Ireland, 2003). MELDER N., HARWIN W. S.: Extending the Friction Cone Algorithm for Arbitrary Polygon Based Haptic Objects. In Proceedings of the 12th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (HAPTICS?04) (2004). [MS05] MARTÍN J., SAVALL J.: Mechanisms for Haptic Torque Feedback. In Proceedings of the First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (2005). [OL03] OTADUY M. A., LIN M. C.: Sensation preserving simplification for haptic rendering. ACM SIGGRAPH 2003 / ACM Transactions on Graphics 22 (July 2003), [OL05] OTADUY M. A., LIN M. C.: Stable and Responsive Six-Degree-of-Freedom Haptic Manipulation Using Implicit Integration. In Proceedings of the First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (2005). [PJC04] POTTER K., JOHNSON D., COHEN E.: Height Field Haptics. In Proceedings of the 12th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (HAPTICS?04) (2004). Submission id: 45 / Rendering detailed haptic textures