Mobile Telepresence Services for Virtual Enterprise

Transcription

1 Mobile Telepresence Services for Virtual Enterprise Petri Pulli, Peter Antoniac, Seamus Hickey University of Oulu - Department of Information Processing Science PAULA Project sponsored by Academy of Finland P.O. Box 3000, FIN Oulu, Finland; Petri.Pulli@oulu.fi, Peter.Antoniac@oulu.fi, Seamus.Hickey@oulu.fi Abstract The advantages of contemporary communication and information technologies based on virtual reality techniques offer for companies a totally new approach to cooperation. Use of virtual product models can be extended to product development, manufacturing, marketing, customization and selling of products for Internet based virtual enterprises. The presentation addresses how the virtual product models and personal mobile telepresence may be utilized as means of communication between product design, manufacturing, marketing and sales. In this paper we offer an approach to adding aspects of distribution, mobility and augmented reality to telecooperation in virtual enterprise, discuss the technology and potential future product concept vision, the CyPhone, and some user scenarios and potential services. Finally, a survey about relevant telecooperation services is presented. In addition to various telepresence based services also several augmented reality based services has been identified with potential to enhance groupwork in co-located teams.

2 1 Internet Virtual Enterprise Concept Electronic commerce is expected to reduce coordination costs in the value chain of manufacturing and sales of products and thus holds promise to increase trade volumes. This can already be seen to take place in the Internet markets with relatively simple products. In this paper the prerequisites for electronic commerce for non-bulk, value-added intelligent products, such as electronics and telecommunication products, are analyzed. When compared with bulk products, there are many challenges in electronic commerce for such intelligent products, as product search and evaluation, customer needs and product matching, product configuration, and agile manufacturing of differentiated products based on customer order. Virtual reality technique in product marketing, customization and selling might be used, for instance, in: Providing an Internet virtual salesroom where potential customers may experiment various product features and ultimately also make product purchases. Interactive customization of products through Internet before ordering and delivering, e.g., color and shape of products may be selected from a predefined set of alternatives, customer may also pick up options from the list of additional features or even propose new features. Collecting user preferences in "customer clinics" for guiding development towards products with higher customer appeal and satisfaction. Presenting and launching leading-edge products prior to physical products being available through manufacturing and sales channels. A scenario of the different ways of utilizing virtual product models in the various phases of product development, manufacturing, marketing and sales is described below (cf. Figure 1 Virtual product models in the context of the value chains of a virtual enterprise.) in the context of the different value chains connected to the virtual enterprise [1]. Virtual prototyping models offer an entirely new perspective for electronic commerce on the Internet since they provide a strong sense of reality and the means to test the model of a product through meaningful interaction. All-digital prototypes resemble their physical counterparts as close as possible in terms of visual three-dimensional images, behavioral, haptic and auditory characteristics. On-line trial of the goods could also be offered, which means that the customers browsing in an Internet salesroom can test operations of a virtual prototype which is simulated like its physical counterpart.

3 VIRTUAL DESIGN ROOM Product Idea VIRTUAL TESTROOM (OF COMPONENTS) Series of Evolving Virtual Product Models VIRTUAL TESTROOM Exterior Mechanics Production Design SW Design HW Design Digital Models of Components Design Component 1 Design Production Purchase of Component 1 Digital/Virtual Product Models in Product/Production Development Design Component 2 Design Production VIRTUAL SALES- ROOM Digital/Virtual Product Models for Marketing/ Sales Support MARKET MECHANISM Purchase of Subcomponent Manufacture of Component 1 Subsystem 1 Assembly Hierarchy of in-house development and outsourcing VIRTUAL ENTERPRISE Final Assembly Marketing and Sales Channels MARKET MECHANISM Figure 1 Virtual product models in the context of the value chains of a virtual enterprise. 2 Virtual Product Models A virtual world is a computer generated, often three-dimensional environment in which a user has a sense of presence, where she or he is able to navigate around and can interact with the objects included in it. A virtual prototype is defined [2] as a computer based simulation of a prototype system or subsystem with a degree of functional realism that is comparable to a physical prototype. Virtual prototyping means the process of using virtual prototypes instead of or in combination with physical prototypes, for innovation, test and evaluation of specific characteristics of a candidate design. The following aspects are dominant: Functionality: The intended functionality of the prototype that is created virtually is clearly defined and realistically simulated; e.g., product functionality and dynamic behavior. Human Interaction: If human action is involved in the intended functionality of the prototype, the human functions involved must be realistically simulated, or the human must be included in the simulation; i.e., real-time operator-in-the-loop simulation.

4 Environment: If no human action is involved in the intended functionality of the prototype, then either an off-line (non-real-time) computer simulation of the functions can be carried out, or a combination of computer (non-real-time) and hardware-in-the-loop (real-time) simulation can be carried out. In virtual prototyping, a virtual world comprises both the VR model of a product under development, i.e., virtual prototype, and the model of the product's target environment, called virtual target environment. Virtual worlds, the VR models, are highly advanced humancomputer interfaces based on Virtual Reality techniques, providing a strong intuitive sense of reality and the means to change the world through meaningful interaction. Virtual worlds may include various levels of "reality", thus enabling us to concentrate on any desired aspects of the product. The interaction between the user and the virtual world can be based on different kinds of interaction interfaces (see Figure 2). The most elementary interaction interfaces can be a keyboard and a mouse with a conventional window-based 2-D user interface. A more sophisticated virtual prototyping environment with a 3-D user interface may include, e.g. a head mounted display, a 3-D position/orientation tracking devices, auditory, and haptic feedback devices. The type of representation in a virtual prototyping environment may be spatial, n-dimensional with abstract information spaces, or various combinations of video, natural and computer-generated images. During the development process, a virtual world can be seen as a model that best matches reality. Figure 2 Functionality of typical virtual product models for intelligent consumer electronics and telecommunications products.

5 Virtual prototype worlds might be used, for instance, in: Universal combining of different models to improve communication in development projects. Prototyping the user observable behavior and the look/feel aspects of the product that are difficult to specify explicitly. Collecting user preferences in "customer clinics" for guiding development towards products with high customer appeal/satisfaction. Providing early visibility of a new product in order to demonstrate its capabilities and features. Modeling systems that cannot be implemented using contemporary technology but which are expected to be feasible in future if technology advances at a predicted rate of progress. Providing a virtual salesroom where potential customers may experiment with various product features and ultimately also make product purchase. 3 Distribution and Mobility A necessary aspect of the full exploitation of the results of virtual prototyping is the requirement for the communication of the results to the interested parties. These may include persons taking part in actual product development - digital signal processing designers, software and hardware designers, and mechanical designers - or people involved in the sales and marketing of the product - marketing and sales representatives, wholesale and retail representatives, and of course customers. Quite often these persons may be quite widely geographically distributed, so solutions based on utilizing communication networks are needed. Several networked virtual reality environments [3, 4, 5, 6] exist today, but none of them supports mobility of users. Most of the networked environments are based on Internet, which could be easily replaced by a mobile multimedia wireless network like SWAN [7] providing mobile connections. However, typical currently available applications and their interfaces, based on immersive virtual reality and heavy desk-top computers, would still restrict the user s ability to move and access services in a natural and convenient way. In the Nara Institute of Science and Technology in Japan [8] an experimental mobile virtual reality system

6 is being developed. This system like ours [9] is based on augmented reality merging both real and virtual environments to provide totally new services and interfaces to mobile observers. Our current work is demonstrated by imaginary mobile telepresence and augmented reality terminal concept "CyPhone" in Figures 4-9. Please note that CyberPhone does not exist yet, it is just a vision of the future, and we expect that it can be real around CyPhone can serve as a product platform for many potential value-added services for telecooperation in virtual enterprise. We have considered services falling in to following categories: Telepresence services (distributed meetings, distributed reviews, teleteaching) Annotation services (guidance, electronic commerce) Monitoring & maintenance services (real-estate and property maintenance and alarm systems) Figure 3 A logical view of telepresence meeting with a chairman and two company representatives physically present

7 Figure 4 Physical representatives of the telepresence meeting Figure 5 Chairman s view of the telepresence meeting through augmented reality eyeglasses

8 Figure 6 A participant transmitting and his 2D mobile phone view. Figure 7 The CyPhone User Interface with a touch pad and a display screen. It has a panoramic camera for 360 video broadcasting.

9 Figure 8 Augmented reality eyeglasses are an optional CyPhone accessory. They can be used for more convenient viewing visual information. Figure 9 An example of using augmented reality for guidance and navigation service. A major challenge is synchronization of real and virtual worlds. In this case direction annotations.

10 In the Personal Access and User Interface for Multimodal Broadband Telecommunication (PAULA) project, funded by the Academy of Finland, we are experimenting with a distributed VR meeting service [11] where the participants are using a radio access network (e.g. WLAN, GSM). The system comprises of a meeting room, in which a number of people are present and to which one or more people will participate remotely. The system is comprised of a VR conference terminal, the remote users terminals, a WLAN to provide mobile access and a supporting service profile and is shown in Figure 10. The VR conference terminal is comprised of an omni-directional camera and a bi-directional 3D sound system.the omni-directional video camera captures a 360 degree live video stream. Software converts this video stream into a fully navigational 3D environment. The bidirectional 3D sound system is comprised of a number of directional microphones. The VR conference terminal has a service profile, which controls the remote users access, security and service privlages. Additional service will enable the remote user to zoom in, record notes or slides. The remote users terminal will be based on the CyPhone concept providing them with a high degree of immersion. While this system has many benefits, most importantly, it affords, we feel, a number of advantages over current systems. It supports complete mobility of the user and the use of a mixed real and virtual world to provide a heightened sense of immersion for a remote meeting participant. Figure 10 A mobile distributed meeting service

11 4 Conclusions We believe telepresence and augmented reality for natural extension to telecooperation in future personal and corporate environments. Wireless broadband local area networks and public mobile networks can provide a sufficient platform for the transmission aspects. The presentation aspects are much more difficult and need innovation and trials. We describe our approach which is based on stereoscopic personal telepresence terminal, Cyphone, and its related user scenarios, part of which are depicted in this paper. Currently we are building research environment and experimenting with a multimodal personal augmented reality user interface (PAULA Figure 11) for CyPhone product concept described in this project. For research purposes we are building also partial implementations using today s technology for some parts of the CyPhone concept. for instance, combinations of omnidirectional optical systems and computing algorithms for stereoscopic image capture, compression, reconstruction and display are under research and experimentation (Figure 12). We are enhancing and using our virtual reality prototyping environment for developing the CyPhone concept further and for demonstrating the capabilities. Hardware prototypes will be built when the level of available technology is mature enough. Additionally some projects are under preparation for elaborating particular user scenarios. One of the first include virtual partly remote meetings, telepresence based multimedia home aid service for senior citizen, and typical traveling business transactions when arriving a foreign town by flight to go for a sales meeting and accessing technical support from headquarters. The concepts proposed in this paper are to be tried and verified through research and field trials in Mobile City Oulu information society pilot program in Finland. We are open and looking for cooperation European 5th R&D framework projects in the areas indicated in this paper.

12 Figure 11 Multimodal personal augmented reality user interface (PAULA Project)

13 Figure 12 Panoramic camera and other VR technologies will bring more in the Future User Interface and displays.

14 6 Acknowledgements This research has been carried out in PAULA, PIHVI, CYPHONE, VIRPI, and VIRPI2 projects (Picocellular based personal virtual products and services) co-funded by Academy of Finland, Technology Development Center of Finland (TEKES), CCC Software Professionals, Elektrobit, Nokia - Nokia Mobile Phones, Polar Electro, J.P. Metsävainio Design, C3 Suunnittelu, and Sonera. References [1] Wigand, R.; Benjamin, R.: Electronic Commerce: Effects on Electronic Markets. Journal of Computer-Mediated Communication vol. 1, issue 3. December, shum.huji.ac.il/jcmc/vol1/ issue3/wigand.html. [2] Haug, E.; Kuhl, J.; Tsai, F.: Virtual Prototyping for Mechanical System Concurrent engineering. In: E.J. Haug (ed). Concurrent Engineering: Tools and Technologies for Mechanical System Design. NATO ASI Series F: Computer and Systems Sciences. Vol Pp Springer-Verlag. Berlin Heidelberg [3] Carlsson, C. and Hagsand, O.: DIVE-a Multi-User Virtual Reality System. In Proceedings of the IEEE Virtual Reality Annual Symposium (VRAIS 93), September 18-22, 1993, Seattle, Washington, USA, pp [4] Shaw, C., Green, M., Liand, J., and Sun, Y.: Decoupled Simulation in Virtual Reality with The MR Toolkit. ACM Transactions on Information Systems 11(3), 1993, pp [5] Snowdon, D. and West, A.: The AVIARY VR-system. A Prototype Implementation. In Proceedings of the 6th ECRIM Workshop, June, 1994, Stockholm, Sweden. [6] Tarr, R. and Jacobs, J.: Distributed Interactive Simulation (DIS). In Proceedings of the 1994 Summer Computer Simulation Conference, July 18-20, La Jolla, California, USA. [7] Agrawal, P., Hyden, E., Krzyzanowski, P., Mishra, P., Srivastava, M., Trotter, J.: SWAN: A Mobile Multimedia Wireless Network. IEEE Personal Communications, 3(2), 1996, pp [8] [9] Pyssysalo, T., Kinnunen, U., and Pulli, P.: Real-Time Distributed Virtual Reality. In Proceedings of the 2nd International Workshop on Mechatronical Computer Systems for Perception and Action, February 10-12, 1997, Pisa, Italy, pp [10] Tuikka T. WebShaman [ [11] Maxfield J, Fernando T., Dew P.A Distributed Virtual Environment for Concurrent Engineering. Virtual reality Annual International Symposium 95, North Carolina,USA, IEEE Computer Society Press