A Classification for User Embodiment in Collaborative Virtual Environments

Transcription

1 A Classification for User Embodiment in Collaborative Virtual Environments Katerina MANIA Alan CHALMERS Department of Computer Science, University of Bristol, Bristol, UK ( Abstract The general goal of Collaborative Virtual Environments (CVEs) is to provide a space within which people may interact. CVEs are increasingly being used to support collaborative work between geographically separated participants. User embodiment is concerned with the provision of users with a representation of their choice so as to make others (and themselves) aware of their presence in a virtual space. The taxonomy presented in this paper details many of the existing networked virtual environments and examines the fundamental interaction interfaces which these systems provide. By initially discussing the features of communication which should be supported regardless of the medium available, the following investigation reveals an incomplete support for non-verbal communication cues over the range of the environments examined. 1. Introduction Most of the networked virtual communities have until recently been text-based. However, such environments are now increasingly using 3D graphics to represent the space and people that inhabit them in real-time. The users, after being connected to a networked system, choose a graphical representation of themselves, termed an embodiment or an avatar [15]. They can now explore the environment by controlling their graphical representation and also interact with other avatars. Although these systems have evolved graphically, communication is still predominantly based on text or audio links. User embodiment is there to indicate the presence of the user in a particular location while the interface provides limited support for non-verbal communication features[16]. This also affects self-representation of the participants in a CVE and the capabilities of personalisation of their selected body images; these images should convey information about the identity, the personality or even the availability of each user. The basic premise of this paper is, therefore, that incorporating such fundamental behavior and features is crucial for the credibility of the virtual interaction. 2. Background The first application of networked computer graphics appeared in 1972 on ARPANET, the computer network developed by the Advanced Research Projects Agency[13]. This network was mainly intended for co-operative work and for sharing information. Today, multi-user virtual environments are used for a variety of purposes, including shared scientific visualization, training, co-operative work, battlefield simulation and entertainment games. Several platforms exist for building multi-user virtual worlds, some of them free and easily accessible through the Internet. Obviously, the performance of these systems is different from high-end applications which are specialised, expensive and mostly running on dedicated networks. Although this gap is shrinking, the future of networked environments which are able to accommodate a large number of users and provide complex interfaces and rich user embodiments depends on aligning a number of

2 technical (networks, computer graphics capabilities, etc.) and social issues (telephone companies, government regulators, etc.)[14]. 3. Non-Verbal Communication New media such as distributed virtual environments, force researchers to analyse what is fundamental about communication[17]. We follow Abercrombie[1] in thinking of conversation as relying on all channels of communication through which information is exchanged by individuals during face-to-face interactions. Language is closely linked with and supported by, non-verbal communication which adds to the meaning of utterances, provides feedback, controls synchronisation and also plays a central role in human social behaviour[2]. Facial expressions: The face is one of the most important areas for non-verbal signaling. In general, facial expressions are indicators of personality and emotions, serving also as interaction signals[8]. Facial expressions provide feedback and information about the listener's level of understanding while revealing interest, puzzlement or disbelief. In addition, affective expressions allow listeners to infer the speaker s current emotional state and communicate their audience's emotional reaction to what is being said. Gaze: Gaze[3] is a general indicator of attention and can be directed at other conversational participants in face-to-face interaction as well as at features of the physical environment. Gaze is closely coordinated with verbal communication. It is used to obtain feedback on the other's responses while talking and extra information about what is being said while listening. In addition, shifts of gaze are used to regulate the synchronisation of speech. Gaze is also used as a signal in starting encounters, in greetings, as a reinforcer and to indicate that a point is understood. Gestures: The hands and to a lesser extent the head and feet can produce a wide range of gestures. Gestures are closely coordinated with speech and support multiple communication functions. They are used to co-ordinate conversational content, achieve reference and assist in turn taking. Conventional gestures are usually intended to communicate and are normally given and received with full-awareness. Posture: This is the information supplied by the orientation of a conversational participant's body. Posture is an important means of conveying interpersonal attitudes and is associated with emotional states. Posture accompanies speech in a way similar to that of gesture and provides feedback to the speaker about how the message is being received. Body position and orientation can also be used to include or exclude people from the conversation. Self-Representation: Self-representation can be regarded as a special kind of non-verbal communication. In general, the main purpose of manipulating appearance is to send messages about one-self. Thus, people send messages about their social status, their occupation, their personality or their mood. Appearance is also used to signal attitudes towards other people for example, aggression, rebelliousness and formality. Bodily Contact: Physical touch seems to have a primitive significance of heightened intimacy and it produces increased emotional arousal. Some forms of bodily contact are used as interaction signals like greetings and farewells or as attention signals. However, the precise meaning of a particular form of touch depends on the culture. 4. Platforms DIVE: The Distributed Interactive Virtual Environment(DIVE) is an internet-based multiuser virtual reality system developed by the Distributed Systems Laboratory of the Swedish Institute of Computer Science(SICS)[7]. DIVE supports the development of

3 shared multi-user virtual environments, user interfaces and applications. The DIVE platform is experimental and available for free for non-commercial use. A participant in a DIVE world is either a human user or an automated application process and is represented by an avatar. The simplest form of embodiment in DIVE is called 'blockie' and consists of a set of primitive 3D boxes which give a sense of presence and orientation[5]. In some DIVE applications, more sophisticated humanoid avatars are used, combining the use of texture-mapped photographs in order to give a stronger sense of identity. Personalisation is available through a set of default options. Embodiments in DIVE have the capability of head-movements, thus directing gaze while navigating. If a user is absent, the relevant embodiment is moved below the ground plane. Touching or grabbing is allowed and this may be used to determine, after a possible angry reaction, if a user is there[5]. DIVE supports audio as well as the display of real time video streams. MASSIVE: MASSIVE(Model, Architecture and System for Spatial Interaction in Virtual Environments)[11], a laboratory prototype from the University of Nottingham, UK, is a virtual reality conferencing system which scales to large numbers of participants. Its users interact in the same virtual world through a variety of different equipment, media and user interfaces(2d, 3D, text, audio)[6]. As in DIVE, the simplest embodiment is a 'blockie'. MASSIVE encourages the use of different colours for the avatars and name labeling to strengthen identity. Users who are connected through the text interface, are represented by the first character of their names. Each embodiment varies according to which medium each participant uses to connect to the system (if a user is connected through a textinterface, the user has a 'T' embossed in his/her head, an audio-capable user has ears, etc.). MASSIVE allows users to personalise their embodiments, however, limited practice of that capability has been recorded because of a shortage of modelling tools. Head movement capabilities are available as well as a selection of simple pre-programmed gestures such as sleeping (which is also used to indicate the user's presence) and blushing. VLNet: VLNet(Virtual Life Network) is a networked virtual environment developed in the MIRALab of the University of Geneva and the Computer Graphics Lab of the Swiss Federal Institute of Technology[12]. The system uses 3D human figures for avatar representations. The VLNet creators divide virtual humans according to the methods used to control them: directly controlled where face and joint representation is modified using sensors attached to the user's body; user guided where the user defines tasks for the embodiment to perform; and, autonomous that are self-governing and incorporate internal states of actions. In particular, an attempt was recently made to incorporate a non-verbal communication interface in the VLNet system. The interface includes two main windows, the posture panel and the gesture panel. Each panel consists of buttons displaying the actual action available and a textual label. The posture panel includes one section for the body (neutral, attentive, determined, relaxed, insecure, puzzled) and one section for the face (neutral, happy, caring, unhappy, sad, angry). The sections are divided into three columns: positive, negative, neutral. The gesture panel consists of one section for the head/face (yes, no, nod, wink, smile), one for the hand/arm (salute, mockery, alert, insult, good, bad) and one for the body (incomprehension, rejection, welcoming, anger, joy, bow). Other methods were explored for integrating facial expressions in VLNet such as video texturing of the face, model-based coding of facial expressions and lip-movement synthesis from speech. dvs: This is the commercial VR system by DIVISION Ltd., UK, which supports multiuser VR applications. A user can incorporate in the system any preferable embodiment (3D geometry) which is defined in the registry file of the platform on start up. If a 2D mouse is used to navigate in the world, then the whole body will follow the viewpoint around. If the user is wearing a head-mountain-display and uses a 3D mouse then the visual embodiment will represent the movement of the tracked joints; inverse kinematics are available to represent elbow position for example. If the user movements are tracked with

4 head and arm sensors then the user can gesture as in the real world with these limbs. In addition, if the user wears a cyberglove, then finger gestures could be tracked. Voice can be used to transmit emotion. Active Worlds: Active Worlds is a client/server software developed by Circle of Fire Studios Inc. The Active Worlds browser (client software), which is free, allows the user to move around the 3D universe and navigate from one world to another. In addition, the Active Worlds server which allows the user to own land in the Active Worlds Universe is free for a 30-day trial period. There are also servers (not for free) which can be used to build independent multi-user shared environments. The software provides a selection of humanoid avatars of various sexes and ethnicities for the user to choose from. The avatars communicate by means of text which is displayed in the text window as well as on top of the avatar that 'speaks' for 30 seconds or until the next message appears. Messages from the closest 12 users are displayed. The interface also provides some predefined action buttons on top of the world window such as 'happy', 'angry', 'wave', 'jump', 'fight', 'dance'. Each body executes a distinctive set of idle motion sequences randomly; for example, some avatars check their watches once in a while. Blaxxun: Blaxxun was the first company to produce a multi-user VRML compliant client, called Cybergate. The Blaxxun Community Server evaluation copy is free and it enables multi-user web capability for three users. The avatars can communicate through a text window which is placed below the 3D world. The system provides a set of cartoon-like avatars. There are eight gesture options which are displayed as buttons under the text window or activated in the text area using the G button: Hello, hey! dislike, no, not, bye. Participants can also provide as much information about themselves as they wish; different levels of privacy are available. Users may also define their own custom avatars by changing the initialisation file of the world. These avatars can also incorporate the default gestures by altering the avatar VRML file. OnLive!: OnLive! technologies is a company that offers commercial multi-user voice client/server software that enables groups of people to communicate with their own voices through the Internet. The participants use 3D embodiments which consist of only a head without a body[18]. The OnLive! worlds require the use of the OnLive! Traveller browser in order to visit the world. Once inside, a participant simply speaks via a microphone connected to the PC in order to engage in conversations with anyone in the environment. As in a real room full of people, chat participants who are close seem louder than those whose avatars are further away in the space. There is also a text-based interface which allows the user to choose an avatar through a pull-down menu and type a message. Eye blinking, lip synch, basic face layout and four basic expressions (happy, sad, surprise, anger) are available. Initially, the user chooses an avatar from two pre-defined sets which include three groups of heads to choose from: animals, fantasy characters and people. In addition to changing the emotional state of each selected avatar, users can change the colour on groups of polygons, as well as their size and shape. Avatars may be also personalised through voice modification and disguising options. OZ Virtual: OZ Interactive develops Internet software and applications that enable realtime collaborative communications in shared spaces on the Internet with a strong focus on creative content production. The company released Oz Virtual, a 3D world viewer that plugs into Netscape Navigator or Internet Explorer and is based on proprietary technology. The multi-user capabilities come via servers which are not commercially available. Oz developed its own format which is a blend of VRML and Oz's own motion control format while audio is supplied by Voxware. Oz Virtual comes with an avatar editor that offers selection from a set of pre-defined avatars and then allows the user to modify their appearance. When the user chooses the initial avatar, there are different sections of modifications that can be applied. The first one comes with the label 'Pieces' which

5 presents the user with a list of avatar parts that can be modified: torso, head or feet. There is no choice for arms. Colours of all the body parts of the avatar can be changed and scaling can be set for any of the three body areas. The last section is labelled 'motions' and provides a set of gestures, steps or dances. There also some expressions available as well as movement of eyelashes and a lip-synch mechanism. All Oz avatar movements are created using motion capture data. Oz also built an intranet environment for Ericsson incorporating avatars with animated facial expressions that could even use mobile phones inside the virtual world[18]. Community Place: Sony offers free multi-user server software as well as a browser and an authoring tool. Community Place is based on the VRML specification. Certain objects in the world have pre-programmed "actions" and users can share the experience of manipulating them. A chat window is also provided. In a Community Place world, the user chooses an avatar from a set provided by the author of that world. The user can change the colours of certain parts of the selected avatars. However, the browser accepts any VRML custom-made avatar modeled in any authoring tool. There is also a limited set of gestures/postures that are provided although the user with experience in VRML/Java can add different gestures as well. The default posture set includes: normal, hello, smile, wao(excited), woo(rejection), umm(skeptic), bye and sleep [18]. Quake: Quake is a multi-user virtual reality game. The player navigates through texturemapped environments very fast using a humanoid avatar (male or female) initially selected from a default set. Facial expressions are not available, however a set of actions could be triggered using the keyboard: walk slowly, walk, run, jump, crawl, hello, aggression, this way. There is also communication through a chat-line, however, messages reach all the players and not just a selected one. Worlds Chat: Worlds Chat from Worlds Inc. is a 3D social environment where the user can explore individual platforms and rooms on a space station and communicate with other visitors through text. It is the first graphical chat system to incorporate 3D. There is a free demo version of the software with limited features. The full version of the software provides a choice of 40 avatars, capabilities of visitors using custom avatars, unlimited session length. In addition, password protection lets visitors have permanent identity. SPLINE: SPLINE (Scalable platform for Large Interactive Networked Environments) is a multi-user platform developed in the Mitsubushi Electric Research Laboratories. SPLINE provides support for multiple users communicating with each other using natural spoken language while navigation is based on users cycling around the world. Users also interact with computer simulations which range from the very simple (e.g., a revolving door) to the very complex (e.g., a human-like robot). 5. A glance at virtual physical touch Physical touch as a particular strand of non-verbal communication is used when an interpersonal bond is being offered or established. And, similar to a direct gaze, physical contact seems to strengthen other messages, for example persuasion[2]. However, touch also carries the implication of invasion of privacy. Researchers are starting to address the issue of touch and haptics in the virtual world. Some initial experiments were undertaken dealing with the influence of haptic communication on the sense of being together. This work concluded that haptic feedback adds significantly to the sense of togetherness[10]. However, incidents have been reported where personal space was invaded by avatars without permission, concluding overall that there are lots of social conventions that are transferred from the physical world to the virtual world[4]. In general, there is limited support for bodily contact in the existing systems, however, there are many technical as well as social issues to be examined further as this is incorporated into the virtual world.

6 6. Discussion The premise of this paper is that communication is accomplished as a combination of speech/language and non-verbal communication features[2]. In addition, face-to-face interaction is accompanied by involuntary expressions making communication live and more natural. We examined the way that existing multi-user platforms incorporate non-verbal communication and the respective interfaces concerned. Most of the systems provide a limited set of gestures, facial expressions or actions which are activated by mouse clicks on relevant buttons. Although, VLNet, for example, incorporates non-verbal communication cues for the face, the body and the hands, in general, there is a significant overlap between the user's actions and the mouse clicks[9]. The non-verbal communication interface for the text-based systems such as ActiveWorlds, Blaxxun and Worlds Chat, tends not to be used at all because the user is busy typing away in the text window. Active Worlds includes some involuntary expressions and OnLive! as well as Oz incorporate lip-synch, but these actions are automatic and are not directed by the user. Motion tracking, on the other hand, used by systems such as dvs could capture movements or gestures directly from the user but also require specialised hardware. Gaze, for instance, while the user needs to look at the screen, is not mapped correctly. In addition, emotional displays were limited in all the systems. Voice/audio support could help to convey emotion on DIVE, Massive, onlive! and Oz, however, research has showed[6] that troublesome audio could affect turn-taking in a conversation; again, non-verbal communication cues are invaluable. A novel approach was presented by Vilhjalmsson in his thesis[16]. His system, 'BodyChat', treats the avatar as an autonomous agent whose face is animated based on a set of parameters. Still, the avatar is partially controlled by the system and not directly by the user. Having in mind that the interface should be driven by the task, the challenge for the future is to create an interface which incorporates all aspects of non-verbal communication. References [1] Abercrombie D., Paralanguage, Communication in Face-to-Face Interaction, ed. J. Laver, S. Hutcheson [2] Argyle Michael, Bodily Communication, Methuen & Co Ltd, London [3] Argyle M., Cook M., Gaze and Mutual Gaze, Cambridge University Press, UK, 1976 [4] Becker B., Mark G., Social Conventions in Collaborative Environments, Proc. of CVE'98, Manchester. [5] Benford S., Bowers J., Fahlen L.E., Greenlhalgh C., Snowdon D., Embodiments, avatars, clones and agents for multi-user, multi-sensory virtual worlds, Multimedia Systems, Springer-Verlag, 1997 [6] Bowers J., Pycock J., O'Brien J., Talk and Embodiment in Collaborative Virtual Environments, CHI'96 [7] Carlsson and Hagsand, "DIVE - A Multi User Virtual Reality System", IEEE VRAIS, Sept, 1993[10] [8] Ekman P. and Friesen W.V., Unmasking the Face, Prentice-Hall Inc., 1975 [9] Henne P., Mark G., Voss A., Gestures for Social Communication for Virtual Environments, BT Presence Workshop, [10] Ho C., Basdogan C., Slater M., Durlach N., Shrinivasan M.A., An Experiment on the Influence of Haptic Communication on the Sense of Being Together, BT Presence Workshop, [11] Greenhalgh C., Benford S., Virtual Reality Tele-conferencing: Impl and Exp, Proc. of ECSCW'95. [12] Guye-Vuilleme A., Capin T. K., Pandzic I.S., Thalmann N.M, Thalmann D., Nonverbal Communication Interface for Collaborative Virtual Environments, Proc. of CVE'98, Manchester, UK. [13] Norberg Arthur and O' Neil, Judy, Transforming Computer Technology: Information Processing at the Pentagon, Baltimore: John Hopkins University Press [14] Shroeder R., Networked Worlds: Social aspects of Multi-User Virtual Reality Technology, Sociological Research Online, vol.2, no. 4, 1997 [15] Stephenson N., Snowcrash, Bantam Books, 1993 [16] Vilhjalmsson H.H., Autonomous Communicative Behaviors in Avatars, M.Sc. thesis, MIT, 1997 [17] Whittaker S., O'Connaill B., The role of Vision in Face-to-face and Mediated Communication, Video- Mediated Communication, edited by Finn K.E., Sellen A.J., Wilbur S.B. [18] Wilcox S.K., Guide to 3D Avatars, John Wiley and Sons Inc, 1998