COVIRDS: A VIRTUAL REALITY BASED ENVIRONMENT FOR INTERACTIVE SHAPE MODELING

COVIRDS: A VIRTUAL REALITY BASED ENVIRONMENT FOR INTERACTIVE SHAPE MODELING Tushar H. Dani, Chi-Cheng P. Chu and Rajit Gadh 1513 University Avenue Department of Mechanical Engineering University of Wisconsin-Madison Madison, WI 53706 e-mail: gadh@engr.wisc.edu KEYWORDS Virtual Reality, Computer Aided Design, Man-machine Interface, Rapid Shape Generation. ABSTRACT While the current generation of Computer-Aided Design (CAD) systems are well-suited for creating so-called detailed designs, there exists few tools for concept shape design. In this paper the authors present a novel application of VR technology COnceptual VIRtual Design System (COVIRDS) to enhance the design of products and artifacts. COVIRDS allows both CAD experts and non-experts to participate in the design process by virtue of its intuitive and easy-to-use voice and hand input driven interface. With this interface scenario, the human designer can create/modify objects and navigate the design space with the combinations of voice command, hand motions and hand actions in an efficient and intuitive way. For the purpose of real-time editing on the surface and solid models, COVIRDS utilizes optimized higher-level representations while a conventional solid modeler is used as the underlying geometric engine. Preliminary investigations indicate that within the range of shapes that can be currently modeled COVIRDS does provide significant gains in productivity as compared to conventional CAD systems. INTRODUCTION This paper presents the COnceptual VIRtual Design System, COVIRDS, a tool for creating, editing and visualizing three-dimensional (3D) shapes in an immersive Virtual Reality (VR) environment [Dani and Gadh 1997a], [Dani and Gadh 1998]. COVIRDS provides intuitive and efficient voice and hand-based inputs for interacting with product designs. While existing product designs can be directly visualized in COVIRDS, new product shapes are easily created via voice (and/or 3D menu-based commands and hand action/motion in the three-dimensional (3D) space. The research consists essentially of two components: (i) The first focusing on the human-computer interaction (Interface Definition) [Chu et al. 1997], and, (ii) The second on geometry representation for visualization and editing (Shape Representation). The Interface Definition research focuses on identifying the most appropriate interaction modalities (voice/one-hand/two-hand) for various tasks in a virtual reality-based computer-aided design (VR-CAD) environment. The Shape Representation research focuses on developing representations suitable for interactive editing of parametric and free-form shape geometry. INTERFACE DEFINITION A VR interface allows the user to perform activities in the virtual environment in an intuitive manner. While other researchers [Bolt and Herranz 1992] [Kuehne and Oliver 1995] have studied the possibility of using speech and gesture to interact with a graphical representations of shapes and assemblies, there has not been yet a systematical study of the interface requirements for product shape design. The current research investigates the fundamental design tasks for creating shape designs and proposed an effective interface definition for a VR-CAD system. In order to determine an effective interface for COVIRDS, the fundamental tasks for product shape design in a virtual reality based system are investigated. In previous research [Chu et al. 1997], a nodal network that defines the fundamental tasks in concept shape design and the precedence relationships between these tasks has been determined. As shown in Figure 1, there are two essential modes in the nodal network: navigation mode and shape Proceedings of the Industrial & Business Simulation Symposium, 1999 Advanced Simulation Technologies Conference, San Diego, California, April, 11-15, 1999, ISBN:1-56555-167-2, edited by Maurice Ades, pp 108-112. 108

editing mode. The navigation mode provides the designer the ability to navigate through the design space to view the generated geometric shapes. The shape editing mode provides the designer the ability to use voice and hand modalities to create or modify the design shapes. The user performs activities that are represented by nodes in the network. The designer switches between the navigation and the shape editing mode and travels around nodes in the shape editing mode to generate shapes of product designs. The network contains directed arcs that allow the precedence relationships to be stored. Since a given activity can be preceded and followed only by selected activities, the network forms a natural data structure to represent this relationship. For example, the creation activity must be followed by either dimension or location activity. Hand motions (positions and orientations) are determined through the use of six-degree-of-freedom motion trackers attached to the user s hand. Hand motion allows the user to translate, rotate and scale 3D objects interactively. Hand actions allow the user to grasp and release an object in the 3D workspace. A pair of gloves is used to infer the user s hand actions. Once an object is grasped, it follows the hand motions until the hand release it. Examples of hand motions and hand action operations are shown in Figure 2 (a) and (b). Figure 2 (a) shows the operation of using hand motion (translation) to scale the length of a block. Figure 2 (b) shows the operation of using hand action (grasp) to rotate the cylinder. Navigation Mode Shape Editing Mode creation selection dimension location detach delete relationship attach Figure 1. Fundamental Tasks query The fundamental tasks for concept shape design are determined according to the proposed nodal network. The interface tests that use different combinations of interface modalities to achieve typical CAD activities are designed based on the fundamental tasks in each node. For example, the fundamental tasks in the navigation node are zoom-in, zoom-out, viewpoint rotate, etc. Therefore, the interface tests for the navigation tasks contain zoom-in, zoom-out, and viewpoint rotate operations in the tests. The interaction definition in COVIRDS is based on WISL (Workspace-Instances-Speech-Locator) in which shapes are created in a 3D Workspace, by Instancing simple shapes using Spoken commands and interactively Locating them in 3D space using hand input to create more complex shapes. The investigated input modalities are (1) voice command, (2) hand motion, (3) hand action, and (4) locator [Chu et al. 1998]. By using the combinations of these input modalities, the designer can create and modify the design shapes in the 3D workspace. Voice commands are words or phrases issued by the designer through the use of a microphone. The voice recognition program in COVIRDS interprets the designer s commands and takes the appropriate actions. The voice command inputs are mainly used in initiating operations such as viewpoint zoom in/out, object creation/deletion. (a) (b) Figure 2. Hand action/motion A locator is a pointing device containing several buttons. When a locator is used in the design space, a pointing beam that emits from one end of the locator is used to specify a position in the 3D workspace. The buttons on the locator are used to activate or terminate operations such as drag-and-drop, select, attach, scale, etc. For example, in Figure 3, the user points the beam at the object, click down and hold the button to move the object. When the desired position is reached, the user releases the button to drop the object. pointing beam locator object follows locator s transformation Figure 3. Virtual Locator operations Based on the investigation of voice (voice command) and hand input (hand action, hand motion and locator) modalities, a series of tests for the study of the input interface has been performed on COVIRDS. The users perform the voice and hand input operations for a specific set of tasks typical found in creation of shapes of product 109

designs. Subsequently, the users rate each of the input approaches (defined as combinations of different input modalities) according to their preference with respect to easy of use for achieving the tasks. According to the test results, voice commands are intuitive in activating certain operations such as rotating viewpoint, creating an object, deleting an object and attaching an object. The hand input driven locator with its pointing beam and buttons was found to be highly intuitive for navigation, object selection, and in 3D re-location of objects. Hand action and motions were also found to be useful for complex 3D manipulations of objects in the virtual environment. SHAPE REPRESENTATION The difficulty in achieving interactive performance arises from the various computations that must be performed in a VR-CAD application, namely, intersection detection, modeling and rendering. While significant research has been reported on collision detection and rendering methods for VR applications, modeling (including design tools and representations for a VR) has not been researched as extensively. Polygonal faceted representations are typically the representations of choice in VR applications given their suitability for fast rendering. However, such a representation has two disadvantages: (i) It requires a large amount of memory to store a model, and, (ii) Performing editing operations on polygonal model becomes complex (some techniques for manipulation of polygon-mesh based models can be found in [Allan et al. 1989]). A typical approach to overcome these difficulties is to use a dual representation consisting of both a triangular faceted representation and a complete boundary model (via a solid modeler). This is the approach taken in the GIVEN tool-kit [Figueiredo et al. 1991] (Gesture based Interaction in Virtual Environments) and the ARCADE system [Stork and Maidhof 1997]. Thus, while a solid modeler is capable of providing all the functionality desired in a VR-CAD application it is typically not used in such a role. This is because solid modelers tend to be slow and hence are unsuitable given the real-time requirements of VR-based systems. This conclusion is borne out by the well known O(n 2 ) (where n is the number of topological entities; vertices, edges or faces in a given solid) nature of basic operations, such as creation, editing and interrogation of shapes [Mantyla 1988]. Thus, the approach taken in COVIRDS is to use a dualrepresentation scheme, called the Virtual Modeler [Dani and Gadh 1997b]: (a) A conventional solid modeler provides the underlying representation, and, (b) a higher-level graphbased structure (called a Design Intent Graph) is used to store the higher level properties of the shape constructed (such as, the design history, design rules associated with each entity etc.). Some sample parametric parts modeled in COVIRDS is shown in Figure 4 (a) and (b). (a) (b) Figure 4 Parametric part modeled in COVIRDS The ability to rapidly create and edit free-form models is an important requirement for any conceptual modeler, since components often have shapes that cannot be described in terms of typical parametric shapes.. In most CAD systems free-form modeling is done by using the socalled control points that define the surface. The difficulty in using the control points approach is the tricky task of determining where to locate the control points so that the surface takes the form desired. In COVIRDS, a combination of direct and indirect surface manipulation methods gives the designer the greatest flexibility in sculpting new shapes. An example of this approach is shown in Figure 5 in which a square-shaped "protrusion" has been added to an initially flat surface. Such features can either be interactively defined or read from a data file at run-time. Related research includes a system developed by [Furlong 1997] and the MOVE-ON being developed by [Hummels et al. 1997]. Furlong et al. describe a VR system 110

that uses free-form deformation to create virtual sculptures in a VR environment. The MOVE-ON system allows car body styling using two-handed gestures to create and manipulate surface models in 3D and is the most similar to the current research. Ascension TM trackers and emitter Logitech TM Flying Mouse Pair of 5th TM Gloves Crystal Eyes TM LCD glasses and emitter Software Modules: Modeling (Parametric & Free-form) Interaction (Device Drivers, Hand input library, Voice command library and 3D menus) Libraries: WorldToolKit TM (Sense8 Corp.) ACIS TM (Spatial Corp.) Microsoft Visual C++ TM IBM VoiceGold TM Figure 5 Constrained free-form deformation PROTOTYPICAL SYSTEM COVIRDS has been tested in an immersive projectionbased hardware called the Virtual Design Studio (VDS). The VDS consists of a large 8ft x 6 ft screen driven by a stereoscopic projector connected to a Silicon Graphics Octane SSI workstation. An extended range Ascension tracker is used for tracking the location of the designers' hands in space and a PC-based voice recognition system is used for interpreting the voice commands. Figure 6 shows the design facility that has been setup at the I-CARVE LAB for testing COVIRDS on the Virtual Design Studio (VDS). In this environment the designer models part geometry using a glove equipped with a tracker and voice commands. The large screen provides a feeling of immersion that enhances the feeling of presence in the 3D workspace. The large sized screen allows people to participate in the design process. In addition, stereoscopic images viewed by LCD glasses facilitate 3D visualization of the shapes being created thereby enabling the designer and other participants in the design process to gain a true threedimensional perspective of the part being designed. 3D tracking of the hand (via a glove and tracker seen in the figure) and speech input (via a microphone) provide an intuitive and easy to use interface for modeling 3D shapes. Details about the VDS are listed as follow: Hardware: Electrohome Projector Custom Frame (with mirror, screen etc.) Pentium TM Pro PC: SUMMARY Figure 6 Virtual Design Studio This paper presents researches on the development of COVIRDS, a new paradigm for product shape design via virtual reality interfaces. Two major components of COVIRDS are discussed: (1) interface definition and (2) shape representation. Interface definition is investigated to identify efficient and intuitive input approaches for product shape designs in the virtual environment. Shape representation is developed to support interactive creation and editing of product shapes. In this new paradigm, the designer can effectively generate parts by interactively creating and editing objects in the 3D workspace with the combination of voice and hand input approaches. The authors are confident that as more experience in gained with the system and as more improvements are made (based on feedback from users) it will make a significant impact on CAD industry. 111

Further information on COVIRDS can be found at: http://icarve.me.wisc.edu/groups/virtual ACKNOWLEDGMENT: The authors would also like to acknowledge the contributions of Mr. Raj Arangarasan, Mr. Xiaochun Liu and Ms. Lin Wang and Mr. Ning Wang for developing various components of the COVIRDS system. REFERENCES Allan, J.B.; B. Wyvill, I.H. Witten, 1989 A Methodology for Direct Manipulation of Polygon Meshes, New Advances in Computer Graphics: Proceedings of CG International '89, pp. 451-469. Bolt, R.A. and E. Herranz, 1992, Two-Handed Gesture in Multi-Modal Natural Dialog in Proc. of ACM Symp. on UIST, November, 1992 pp. 7-14. Chu, C-C; T. H. Dani and R. Gadh, 1997 Multimodal Interface for a Virtual Reality Based CAD System, CAD Journal, Elsevier Publications, 29(10), pp 709-725., October, 1997 Chu, C-C; T. H. Dani and R. Gadh, 1998 Evaluation of Virtual Reality Interface for product shape design, IIE Transactions Special Issue on Virtual Reality, vol. 30, no. 7, pp. 629-643, July, 1998. Dani, T. H. and R. Gadh, 1997, "Creation of Concept Shape Designs via a Virtual Reality Interface", CAD Journal, Elsevier Publications, 29(7), 1997. Dani, T.H. and R. Gadh, 1997, A Dual Graph Representation within a Geometric Framework supporting Shape Design in a Virtual Reality Environment, Technical Report, I-CARVE LAB, Mechanical Engineering Department, University of Wisconsin-Madison, Sep. 1997. Dani, T. H. and R. Gadh, 1998 Chapter 14: Virtual Reality - A New Technology for the Mechanical Engineer, The Mechanical Engineers' Handbook, 2nd edition.john Wiley & Sons, 1998, Editor: Myer Kutz. Figueiredo, M; K. Bohm, and J. Teixeira, 1991 Advanced Interaction Techniques in Virtual Environments," Computers and Graphics, v. 17, no. 6, pp. 655-661, 1991. Furlong T.J., 1997, Virtual Reality Sculpture using Freeform Deformation, DETC-97/DFM4511, Proceedings of the 1997 ASME Design Engineering Technical Conference and Computers in Engineering Conference, Sacremento, CA, Sep. 14-17, 1997. Hummels C., Paalder A., Overbeeke C. & Stappers P.J., 1997, Gesture-based Two-handed Styling in a Virtual Environment, ISATA Magazine, pp. 11-13, October, 1997. Kuehne, R.P. and J. Oliver, 1995 A Virtual Environment for Interactive Assembly Planning and Evaluation, ASME, Design Automation Conference, Boston, September, 1995. Mantyla, M, 1988 An Introduction to Solid Modeling, Computer Science Press, Rockville, MD. Stork, A.; M. Maidhof, 1997, "Efficient and Precise Solid Modeling using a 3D Input Device," Proc. of the Fourth Symposium on Solid Modeling & Applications, Atlanta, GA, May 14-16, 1997, pp. 181-194. BIOGRAPHY Tushar Dani is a Ph.D. candidate in the Mechanical Engineering Department at the University of Wisconsin- Madison. He received his B.Tech degree in Mechanical Engineering from the Indian Institute of Technology, Bombay, India and his MSME degree from the University of Toledo, Ohio, USA. His research interests include virtual environments for design, computer-human interfaces for virtual reality applications, and Internet-based technologies such as VRML and Java3D. Chi-Cheng P. Chu is currently a Ph.D. candidate in the Department of Mechanical Engineering at the University of Wisconsin - Madison. He received his B.S. from National Taiwan University, Taiwan in 1990 and M.S. from University of Wisconsin - Madison in 1995. His research interests include computer-human interface for virtual reality-based CAD systems and multimodal interface interaction in the virtual environment. Rajit Gadh is Associate Professor of Mechanical Engineering, Director of the CAD-IT Consortium and Director of the I-CARVE Lab at the University of Wisconsin-Madison. Professor Gadh obtained his Ph.D. from Carnegie Mellon University, his MS from Cornell University, and his BS from Indian Institute of Technology, Kanpur. He has been a visiting researcher at University of California, Berkeley for a year, and has 4.5 years of industrial experience in areas relevant to his research. His research interests are in virtual reality-based shape design, shape abstractions for feature extraction in virtual prototpying and geometry-based assembly abstractions. He is the recipient of the NSF CAREER Award (1995), the NSF/Lucent Technologies Industrial Ecology Fellowship (1997), and the Society of Automotive Engineers' Ralph R. Teetor Research an Educational award (1993). 112