More light on your table: Table-sized Sketchy VR in support of fluid collaboration

Size: px

Start display at page:

Download "More light on your table: Table-sized Sketchy VR in support of fluid collaboration"

Rodger Brett Hudson
5 years ago
Views:

1 More light on your table: Table-sized Sketchy VR in support of fluid collaboration Hiroyuki UMEMURO*, Ianus KELLER**, Pieter Jan STAPPERS** *Department of Industrial Engineering and Management, Tokyo Institute of Technology O-Okayama, Meguro-ku, Tokyo Japan, {umemuro, ** Department of Industrial Design, Faculty of Industrial Design Engineering, Delft University of Technology, Landberghstraat 15, NL-2628 CE Delft, the Netherlands, Abstract: Despite two decades of impressive technological improvement, Virtual Reality is only slowly becoming useful in real-world applications for designers. Many VR applications are perceptually overwhelming with high quality graphics and stunning animation, but the support of everyday natural interaction between human and computer, and between humans supported by a computer, is only progressing slowly. Analysis of existing VR technology suggests that the large-scale applications (such as the CAVE and flight simulators) and small-scale applications (such as surgical trainers and miniature holographic systems) lend themselves best for large passive audiences (like a theater with audience) and active single users, respectively. Middle-sized table top systems are probably best suited to support multiple users working together. After all, a table in itself affords use by more people (e.g. eating together). In this poster we compare a number of existing and proposed table-sized VR setups which aim for a high degree of free interaction. Issues of discussion are the need for simultaneous access (multiple users having democratic control over the input/output channels), sketchy interaction (fluidity of expression, allowing suggestive rather than prescriptive information), tangibility (merging virtual and physical), openness (showing clearly to the user what parameters the system is sensing), and rapid implementation (flexibility in how quickly applications can be supported using). We also discuss the importance of aesthetic and technical factors in the expression of interfaces, arguing that most often, virtual reality applications have been used as advanced visualisation systems. An important element is projecting on top of the table from above. Especially for designers involved in interaction design, with its high emphasis on usability and experience, and Kansei factors, VR can be a tool for quickly exploring new interaction styles. In order support these designers, VR tools must enable flexible, rapid, and suggestive implementations of virtual and augmented interface designs. Key words: Virtual Reality, collaborative design, body-scaled interaction, direct manipulation, design prototype 1. Introduction Despite two decades of impressive technological improvement, Virtual Reality is only slowly becoming useful in real-world applications for designers. In order for Virtual Reality to become useful for designers, we need to look at it from a completely different angle, relying on the designer s ability to create, discuss and explore complex problems, in stead of relying on technology and visualization to solve these problems for the designer. Though many VR applications are perceptually overwhelming with high quality graphics and

2 stunning animation, their support of everyday natural interaction between human and computer, and between humans supported by a computer, is only progressing slowly. The early visions of Virtual Reality systems as a technology that supports people at using their natural skills in a simulated environment, which seamlessly connects to their everyday experience, is still far out of reach. One major problem is the high entry threshold to VR, formed by the necessity of complex hardware and complex software. For instance, simple 3D manipulations such as unscrewing the lid off a jar, is typically implemented using two sensorgloves with some haptic feedback and 6D magnetic sensing; the user is encumbered by electrical wiring, and mistakes are made because these systems are hard to calibrate precisely. Moreover, the user cannot correct visually, because he is visually shut off as his VR goggles show the simulated world and exclude light from the real environment. Only with the most advanced systems can such everyday tasks be performed in a natural way. Moreover, there are also software problems, in that almost all VR simulations require extensive custom programming. For these reasons, VR systems are not yet suited for the early, explorative phases of design, although they can be very useful already for less interactive applications such as visualising endproducts. A VR system that supports the early design phases should have a very low threshold for exploring interactive simulations. In our teaching experience, industrial design students are good at conceptualizing VR interactions, but often it can take weeks or months of programming before the interactions can be first experienced. By that time the designer has progressed to other projects. We find there is a need for interactive simulation systems that combine the ease of use of some traditional media with the interactive and dynamic opportunities of VR technology. The answer to that need is not just a simpler programming interface or a dedicated simulation software architecture. We argue here that a solution comes from focusing on the activities of the users, and using a little bit of technology there where it enhances the existing strengths of existing skills and traditional media. We do not claim this is the only way forward, but that designers can benefit from a sketchy form of VR, which stays inbetween on the one hand the totally free conceptualization activities of role-playing and cardboard prototyping, and on the other hand the formalized communication activities of scenario-writing, storyboarding, and systems implementation. 2. Virtual Reality and design tools In this paper, we use the term Virtual Reality (VR) to mean advanced, multi-sensory, input and display techniques creating a synthetic environment in which the user can leverage the skills he or she acquired in the natural environment. This definition is intentional, indicating a striving for what we want the user to experience and do, rather than technological, as would be defining VR through the means of particular hardware, such as a CAVE, a Head-Mounted-Display (see below) or other means of extending the users body by virtual means. Besides these by now archetypal examples of VR, this definition also includes related techniques such as media-spaces (highly interactive large 2D projection systems, see e.g., [11]), augmented reality (mixing computer graphics and physical objects, [1]), and flight simulators. From a user-centered viewpoint, these different technologies pose similar problems, namely, catering to the user s spatial and interactive skills. From a technological viewpoint, there are signs that they are already merging, and radically changing the current interaction paradigms which, even for VR, are still designed around the mouse-and-pointing paradigm ([1],[9]). In

3 this section we present considerations for low-threshold VR use for interactive collaborative design. The issues we address are the need for sketchy, expressive displays, non-monopolized interaction, augmented interfaces mixing virtual and real world elements, and real-time collaboration at a distance. We discuss these issues also in the context of close-to-practice research through design, i.e., we discuss prototypes that can be tested in real-life design tasks by designers, not partial solutions which can only be tested in clinically reduced laboratory tasks. The latter type of projects are important, but too often run the risk of missing the aim of true usefulness for the designer. 2.1 Body scale The biological and psychological space of experience and action is not scale-free such as the space concepts of Euclidean geometry and Newtonian physics which underly most of our technology and scientific thinking. Typically, we use our fingers and hands in precision tasks in a small region. We use our hands and arms for layout actions within approximately a tabletop range. The space beyond that is important for our orientation and situational awareness, but we need special products, i.e., remote controls, for action-at-a-distance. An encompassing VR system should support the user at all three scales, but existing VR systems currently only support one of these scales at a time. [7] describe the TRI system (short for Three Ranges of Interaction ) developed at TU Delft, in which these different ranges are addressed. TRI implements a separate 2D interface for each of the three ranges illustrated in Figure 1. TRI is shown in Figure 2. The large range interface is a curved, panoramic screen, the medium range interface is a tabletop projection, and the small range is a WACOM pen tablet. In this paper we limit ourselves to the medium range, that of hand and arm movements, because it is technically the easiest one to address, and psychologically an important one for designers, as it is the scale at which traditional sketches, models, and plans are made, manipulated, and viewed. Figure 1. Body-scaled partitioning of space. From left to right: detailed manipulation in a close range, layout and placement in the middle range, and overview and context in the large range.

4 Figure 2. The TRI system offers separate interfaces for the three scales of interaction. 2.2 Holography versus sketchiness Designers s tools for the early phases of a design process are characterized by fluent interaction, sketchy expression, and the juxtaposition of many elements (e.g., [10], [12]). Figure 3 shows an example of a compound design sketch. Design support tools have, until recently, provided very little or no support for expressive vagueness. Notable exceptions are expressive rendering filters for 3D CAD (which unfortunately still require non-sketchy creation of the CAD model), and the sketchy 3D CAD program Teddy described in [4]. A design space for interaction should similarly not enforce a rigid, totally structured, interface, but allow for suggestive expressions, graded emphasis, incompleteness, and improvised use of its means. Figure 3. A conceptual design sketch typically consists of a loose juxtaposition of different sketch elements of various visual styles, with deliberate vagueness helping the designer focus on certain aspects of the design, and lowering the threshold for readers to provide constructive criticism. In the TRI system, sketchiness is promoted by beaming light from the front on informal surfaces such as cardboard or tracing paper. The display surfaces are made non-rectangular, to emphasize the nature of informal sketches, and to avoid a pixel-screen like appearance, thereby encouraging the designer using the system to create informal visualisations with an emphasis on content rather than form. Moreover, TRI uses front projection on purpose. With front projection, light falls on the body of the user, and on objects, when they are on or over the table. Such shadows are often regarded as technical imperfections, as errors to be reduced or eliminated in traditional VR systems. In TRI we acknowledge these effects as signs that the user and his body are part of the

simulation, and we promote creative use of these shadows. In a range of student projects, various uses of these effects have been investigated.

By taking a picture of the designer s hand while handling a model and projecting it in the same place.

Figure 4 Left: with front-projection, the shadow of the hand can be used as just like a mouse (registered by a camera below the table; see also Figure 6).

5 simulation, and we promote creative use of these shadows. In a range of student projects, various uses of these effects have been investigated. In one project students used shadows as input device for kitchen stove configuration test. In another project a top view camera was used to bring in the hands of the designers in their own sketches. By taking a picture of the designer s hand while handling a model and projecting it in the same place. By using pen and paper on the same place and in the same scale the designers bring back their hands in the design process. Figure 4 Left: with front-projection, the shadow of the hand can be used as just like a mouse (registered by a camera below the table; see also Figure 6). Right: projecting views from a camera that is aimed down toward the table. 2.3 Augmented interfaces No matter how much effort we put into our visualizations on computer screens or print outs, they never match seeing the real model. Currently we use our models only for the basic form and color and we go into prototyping a real working or interactive products with buttons, screens and switches in a different phase. Using projected light we are able to quickly and exploratively project our interfaces, buttons and layouts on our rough models before we go into a working prototype. We do not use any technological blackbox transformations of our physical models to interact with the system such as 3D-scanning, keeping the mental threshold for the users as low as possible. This system has been successfully used by different masters students in their design projects. Figures 4 and 5 show some example projects. In each case, the software consisted of a small program written in Macromedia Director. The first versions of these prototypes were implemented in less than three days, by design students with little programming experience. In earlier projects, where we used traditional VR programming with 3D graphics, tests of this complexity would take people weeks before anything could be interactively tried out. In the current projects, a lot of time is freed up to experience, explore, and iteratively improve the conceptual design of the sketchy VR implementation using both software techniques and physical media such as cardboard, mirrors, felt tip pens, etc.

6 Figure 5 The material expression of the foam model is changed by projecting textures over it. In this use of the system, the models stand on a glass table, which heightens the convincingness of the simulated appearance, because no shadows are shown. 2.4 Collaboration Most current computer setups, including VR, implement a one-on-one dialogue between a single user and a single computer. In many offices, and design offices are no exception here, a visitor sees people s backs, their heads covering the small screens where they work. The same goes for much Virtual Reality work. As Krueger, one of the pioneers in the field of Virtual Reality, concluded in [8]: While Virtual Reality was once billed as the disappearing interface, a quick glance at a goggled, gloved, suited, and tethered participant makes that claim seem like self-parody. The heavily built-in test subjects of high-end holographic Virtual Reality are well supported for some spatial tasks, but at the price of being cut off from the real world, and from people they work with. Design work in the conceptual phases of a design project requires a lot of collaboration and communication. Design teams interact in discussing and adapting their designs. With traditional media, collaborative work often occurs on a table, with participants standing around it, and all participants having direct access to the materials on the table. With computer-mediated solutions, the interaction is most often channeled through a single input device, such as a mouse, which is held by one user at a time. The collaboration table setup, developed at the Tokyo Institute of Technology, uses a setup similar to TRI s middle range, but aimed at supporting multiple people at multiple locations. Again, front projection is used, but in this setup, Light Emitting Diodes are used as trackable physical objects. A camera under the table tracks the positions of the diodes, allowing simultaneous tracking of several tagged objects. (In TRI, only very simple shadow interaction has been implemented, not multi-point tracking). By linking several tables through an internet connection, both co-located and separated collaboration was supported. In [14], the system is described and a user-test is reported. Observation of the user s behavior with the setup shows that collaborating users can work in parallel, without taking turns, and that participants retain an overview of what is happening on the table.

7 Figure 6. Two collaboration tables linked by a network connection. 3. Discussion and conclusion The approach to VR applications focusing on interaction and overall experience, as opposed to visualization and impressive renderings has resulted in the two setups for tabletop collaborative interaction, introduced in the sections above. Scheme 1 gives an overview of the effects of these solutions in our VR Setups. Body Scaled Interaction By involving the whole body, arms and posture in the interaction we enable more active involvement and expressivity Sketchy aesthetics Allowing for improvised interaction and suggestive visualization evokes active interpretation and explorative use Augmented interface Projecting computer images on physical objects mixes the real with virtual without requiring extensive calibration or confusing our users Collaboration The horizontal plane and scale of the table projection suggests collaborative use and allows for overview and parallel work Scheme 1. The criteria for the VR Setups and the effect they have on their use. The idea of tabletop interaction itself is not new. Several systems have been built in research labs around the world, and have been tested with applications in fields such as city planning and architectural design (see e.g., [2], [3], [5], [15]). As yet, not many systems have been used for supporting concept development in industrial design engineering. Still, the above examples indicate that by using computers and beamers as augmented versions of traditional media, new ideas for computer-supported tools can be explored with relatively unsophisticated means. In fact, a low level of technical sophistication can sometimes be a critical factor for the success, by lowering the acceptance threshold of users, and the development thresholds of system developers. In sketchy design visualisations, realism is consciously avoided so the viewer s attention is focused on the creative process instead of a preconceived goal. Similarly, in Virtual Reality, levels of sketchiness can be introduced to increase user engagement, and to enable designers to quickly explore new and unorthodox tools and techniques. Visual artefacts such as shadows, visibility of the projection surface, or non-rectangular display sizes can be an advantage, just as visible line styles in concept sketches. By avoiding technical perfection, such as holographically precise 3D

8 displays, an extra stage is enabled in which designers can participate in the development and use of VR systems, without being detracted by technical issues such as calibration and programming. References 1. Azuma, R.T. (1997) A survey of augmented reality. Presence: Teleoperators and Virtual Environments 6,4, Buxton, W., Fitzmaurice, G., Balakrishnan, R., Kurtenbach, G. (2000) Large Displays In Automotive Design. IEEE Computer Graphics and Applications 20(4), Fass, A., Forlizzi, J., Pausch, R. (2002) MessyDesk and MessyBoard: Two Designs Inspired By the Goal of Improving Human Memory. Proceedings of DIS 2002, Designing Interactive Systems, Igarashi, T., Matsuoka, S., Tanaka, H. (1999). Teddy: A Sketching Interface for 3D Freeform Design. SIGGRAPH 99 Conference Proceedings, Los Angeles, Ishii, H., and Ullmer, B. (1997) Tangible bits: Towards seamless interfaces between people, bits, and atoms. Proceedings CHI Keller, Hoeben, & Stappers (2000) Aesthetics, interaction, and usability in 'sketchy' design tools. Exchange Online Journal, 1(1), ( 7. Keller, A.I., Stappers, P.J., & Hoeben, A. (2001) TRI: Inspiration Support for a design studio environment. International Journal of Design Computing, 3, pp 17. ( 8. Krueger, M. (1995) When, why, and whether to experience virtual reality. Proceedings of Virtual Reality World 95, Lewis, T. (1997) Absorb and extend: Resistance is futile. IEEE Computer, May, McKim, R.H. (1980), Experiences in Visual Thinking, Wadsworth, California. 11. Simpson, R.S. (1997) Videowalls: The book of the big electronic image. Oxford, England: Focal Press, 2nd edition 12. Stappers, P.J., & Hennessey, J. M. (1999b) Towards electronic napkins and beermats: Computer support for visual ideation skills. In: Paton, R.C. & Neilson, E. Visual Representations and interpretations. Proceedings of VRI 98, Liverpool, September 22-24, pages Berlin, Springer. 13. Weiser M., and Brown, J.S. (1997) The Coming Age of Calm Technology, In P. Denning & R. Metcalfe (Eds.), Beyond Calculation: The Next Fifty Years of Computing, Springer-Verlag, New York, Umemuro, H. (2003) Collaboration table: An alternative medium for multi-user multi-site collaboration. 15. Wellner, P. (1993). Interacting with paper on the DigitalDesk. Communications ACM, 36 (7),

Presence for design: Creating an atmosphere with video collages. Ianus Keller (presenting), Pieter Jan Stappers (TU Delft) and Jorrit Adriaanse (SARA)

Presence for design: Creating an atmosphere with video collages Ianus Keller (presenting), Pieter Jan Stappers (TU Delft) and Jorrit Adriaanse (SARA) Delft University of Technology, Faculty of Industrial