Community Computing: Collaboration over Global Information Networks, John Wiley and Sons, pp , 1998.

Transcription

1 Toru Ishida (Ed.), Community Computing: Collaboration over Global Information Networks, John Wiley and Sons, pp , Chapter 3 FreeWalk: A Three-Dimensional Meeting-Place for Communities 3.1 Introduction Hideyuki Nakanishi Chikara Yoshida Toshikazu Nishimura Toru Ishida Matching the advance of computer networks, various computer systems for supporting collaborative work have been studied. Those systems often provide desktop conferencing tools for the support of business meetings. However, meetings are not always of a business or formal nature. Casual meetings such as chatting at a coee break or in a passageway enrich our life. Though casual meetings also take an important role in collaboration, research has tended to ignore this aspect. We think that conventional desktop conferencing systems, whichmulti- cast pictures and voices, cannot support casual meetings. In such systems, the faces of all participants are always displayed, which strains conversation. Furthermore, those systems hinder many people from participating simultaneously. We have been developing a series of community support systems under the project Socia [Ishida 1994, 1997, Yamaki et al. 1996a]. The project aims at supporting everyday activities by forming a community through computer networks. For example, we have developed a meeting scheduling system,

2 56 Chapter 3. FreeWalk: A Three-Dimensional Meeting-Place which introduced non-committed meeting scheduling, wherein the agents do not commit to any plan and thus do not govern their users' schedule. Free- Walk is our latest product [Nakanishi et al. 1996]. In FreeWalk, a common three-dimensional (3D) space just like a real life park or lobby is oered to enable people to experience accidental encounters in a network. As a result, many people can meet in a more relaxed atmosphere than conventional systems. We describe below the inherent features of casual meetings, and show how FreeWalk can support them. 1. Casual meetings In conventional desktop conferencing systems such as Oce Mermaid [Watabe et al. 1990], participants turn on the system when they start a meeting. Whenever the system is in operation, the faces of all participants are displayed on their workstations, and this hinders free conversation. Since the participants are listed up before the meeting starts, an accidental encounter with an unpredictable participant cannot occur. Several desktop conferencing systems have tried to extend their functions to support casual meetings. CRUISER [Root 1988] randomly selects some of the participants and displays their faces to other participants. This interesting feature, called Autocruise, simulates accidental encounters. In contrast, the approach of FreeWalk is to provide a virtual place for casual meetings rather than promoting any system-directed encounters. FreeWalk provides a 3D virtual space wherein participants can move and meet by themselves to provide maximum freedom to the participants' activity. The faces of participants are displayed only when their bodies meet. 2. Meetings with many people In real life, a meeting often consists of many people. In meetings such as parties, several tens of participants simultaneously exist in the same space. In such cases, it is almost impossible to use conventional desktop conferencing systems that try to display the faces of all participants at once, since it is hard to show the faces of so many people in a screen of limited size, and even if it is possible, it is very hard for users to comprehend the situation. An interesting approach is taken in VENUS [Matsuura et al. 1993], where each workspace is formed as a room, and participants can observe the behavior of other participants through the pilot window of the room. We designed FreeWalk to naturally reproduce human behavior in a 3D virtual space so that many people can meet without confusion. In FreeWalk, participants have locations and view directions and can change them freely according to their own will. They can wander around

3 3.1. Introduction 57 Figure 3.1: FreeWalk Window before they approach and talk to someone else, and they can also watch other participants from any location or view direction. As for the support of meeting with many people, the cost of building the whole system cannot be ignored. For example, meeting systems with special equipment, such as MAJIC [Okada et al. 1994] and the multimedia environment ofatr 1 [Takemura and Kishino 1992], cannot be used by many people simultaneously, simply because such equipment is quite rare and expensive. Our policy in designing FreeWalk is, therefore, to make the system run in commonly used environments. Since social interactions in electronic casual meetings have not been investigated, various experiments are needed for further studies. Though several conferencing systems that use a 3D shared virtual space have been developed [Sony 1997, Sugawara et al. 1994], their major concern is to construct a realistic virtual world. Our approach is, on the other hand, to implement a casual meeting system that runs at practical speed in widely used environments. There exists another application area that uses 3D virtual spaces, namely, videogames. Some of them support multi-user environments wherein users can use their characters to interact with others. After investigating the similarities and dierences between the two areas, we concluded that videogames and desktop conferencing systems can share enabling technologies. 1 Advanced Telecommunications Research Institute International.

4 58 Chapter 3. FreeWalk: A Three-Dimensional Meeting-Place In the rest of this chapter, Section 3.2 shows how people communicate together in FreeWalk, while its detailed implementation is given in Section 3.3. Section 3.4 reports the results of experiments where the performance of Free- Walk is investigated. Section 3.5 performs the functional evaluation of Free- Walk and shows the eectiveness of our approach. 3.2 Interaction Design of FreeWalk D Community Common Figure 3.1 shows an image of a FreeWalk window. FreeWalk provides a 3D community common where everyone can meet others. People can enter the space by specifying the server's IP address. Each participant can move and turn freely in the space using their mouse just as in a videogame. Locations and view directions of participants in the space determine which pictures and voices are transferred. In this 3D community common, each participant is represented as a pyramid of 3D polygons. His/her live video is mapped on one rectangular plane of the pyramid. The participant's view point is located at the center of this rectangle. The view of the community common from his/her view point is displayed in the FreeWalk window. Figure 3.2(a) shows an example view of participant A when three participants A, B and C are located as shown in Figure 3.2(b). B C (a) A's view (b) Map Figure 3.2: Participant's View of Community Common Participants standing far away appear smaller and those near are larger. Participants located beyond a predened distance are not displayed. Voices are also transferred under the same policy. Voice volume is proportional to

5 3.2. Interaction Design of FreeWalk 59 the distance between sender and recipient. Figure 3.3 shows the change in voice volume. Voice volume Figure 3.3: Voice Transfer Simulating Real Life Behavior In FreeWalk, people show up in a common 3D space, wander freely inside the space, and encounter others accidentally. Since the locations and view directions of the participants are reected by pyramid orientation, each participant can watch what other people are doing from a distance. Figure 3.4 shows an example of accidental encounter, where (a) the user nds others on the radar screen displayed at the right bottom corner of the window, (b) watches them to nd out what they are talking about, and then (c) joins them. Even in the meeting, participants can rearrange their locations if necessary. Since each participant is represented as an object in the space, the participant can observe the distances/directions of other participants in the view. He/she can also observe the participants around him/her by turning by his/her head. The spatial distance between participants may reect their mental distance. Figure 3.5 shows the view changes of participants A and B while participant B changes his direction in front ofa. The voice of a speaker is heard by not only the participants of the conversation but also anyone in the neighborhood. Thus, people can join the conversation that attracts their interest smoothly since they can guess the subject beforehand. On the other hand, people can keep a conversation secret by keeping away from others. Voice volume attenuates in proportion to the distance between sender and recipient. Voices are not transmitted if the predened distance limit is exceeded. Therefore, people can form separate meeting groups without bothering each other.

6 60 Chapter 3. FreeWalk: A Three-Dimensional Meeting-Place (a) Find others on the radar screen (b) Watch a talking pair (c) Join their conversation Figure 3.4: Accidental Encounter Organizing Meeting Groups Conventional desktop conferencing systems provide various functions to support the organizational behavior of participants, such as speaker selection. Although these functions enable participants to manage multiple conversation threads in parallel, they also damage the freedom that we aim at. FreeWalk does not take this approach. The common 3D space used in FreeWalk gives a casual feeling to communication. Participants can introduce the various communication styles of their real life into the FreeWalk space. Many participants can simultaneously exist in the same space without confusion, since they can grasp what is going on in the space at a glance. People make a group by standing close to hear the voices and see the faces of each other. Figure 3.6 shows this situation. If there is enough distance between groups, the voices of the people in one group are not heard by people in other groups. This feature makes FreeWalk an eective tool for holding a party with many people. Since the voice is attenuated by distance, a participant must approach the others in order to talk to them. This limitation forces people to combine actions and conversations in the space. People can leave a conversation just by leaving the group, and join a conversation by approaching the group. People who meet each other by chance may start a conversation. This means that

7 3.3. System Design of FreeWalk 61 B A B A B (a) A's view of B (b) B's view of A (c) Map A Figure 3.5: Changes of Participants' View meetings can be started with an accidental encounter. To encourage this, FreeWalk provides a special place with a landmark. Group Group Figure 3.6: Meeting Organization 3.3 System Design of FreeWalk Videogame Technologies We have applied videogames technologies to the implementation of FreeWalk, and we describe here how the technologies were introduced.

8 62 Chapter 3. FreeWalk: A Three-Dimensional Meeting-Place Figure 3.7 overviews various existing communication tools. The X-axis represents the amount of data transmitted, and the Y -axis represents the computation performed by those tools. From the view point of transmitted data, those tools can be roughly classied by data types. Since most chat systems or systems are text-based, their data size is small. On the other hand, desktop conferencing systems send a large amount of live video and audio data. Computation Community Place InterSpace FreeWalk Comic Chat Text Chat Video Conferencing System Transmission Figure 3.7: Overview of Communication Tools The communication tools are also classied by how the transmitted data are presented to users. While systems do not modify data for presentation, Comic Chat [Kurlander et al. 1996], a text-based chat system, generates a comic strip representing the conversation among users. Several multi-user environments with a shared 3D virtual space have been developed. For example, Community Place 2 [Sony 1997] is a text-based chat system implemented on VRML2.0 [Bell et al. 1997]. Some desktop conferencing systems, like InterSpace [Sugawara et al. 1994] and FreeWalk, provide a 3D view of the current meeting status: how many people are in the space, who is talking with whom, and so on. They require more bandwidth for providing live video and audio channels. From Figure 3.7, it is clear that desktop conferencing systems with a shared 3D virtual space are placed at the upper right in the chart. It is worth pointing out that videogames also utilize 3D presentation. Most of them realize 3D virtual spaces so that players can control their characters freely. We think that the following videogame technologies can be introduced into desktop conferencing systems. 2 Community Place is a trademark of Sony Corporation.

9 3.3. System Design of FreeWalk 63 Displaying global situations in a 3D virtual space Most videogames provide facilities for users to grasp global situations, since the characters move rapidly, and their moves are often timebounded. Processing all data in real time The main process loop of videogames consists of checking input data, changing internal states, and updating output data. The processing time of this cycle must be small enough to give users the illusion that the characters in the 3D virtual space are moving naturally. Running on low-cost machines Though some arcade games need special expensive input devices, most videogames, especially those running on game machines for home use (known as \nintendo" machines) or personal computers, do not. People can run them on low-priced general purpose machines with a joy-pad, joystick, mouse, or keyboard. Most of these devices are standard attachments to such machines. The advantages of the videogame technologies discussed above suggest their application to desktop conferencing systems, especially for generating a 3D virtual space and replicating the moves of players' characters in the space. Thus, we applied the technologies of games such as action, shooting, racing, and ghting to the design and implementation of FreeWalk. In videogames, various ecient drawing techniques for 3D spaces are commonly used, since low-cost machines have poor hardware support. One of the most popular methods for constructing 3D spaces is to combine a limited number of polygons [Neider et al. 1995]. To increase realism, a texture image is usually mapped on the surface of each polygon. Without special hardware support like a geometry engine processor or a texture engine, however, the above method cannot achieve real-time performance. The frame rate for drawing the 3D space determines whether users can move freely or not. From our experience, to move freely in the space, the frame rate should be at least 8 to 10 frames per second. Thus, we introduced some common gaming techniques in the implementation of FreeWalk. For mapping a texture image on the ground of the 3D space, the texture image is rst rotated so that the direction of the user's view projected on the ground is upward (see Figure 3.8(a)). We then determine the start point and zooming ratio of every scanning-line according to the angle of depression (see Figure 3.8(b)), which imitates the perspective representation. Since the method is simple and ecient, it is often used by existing videogames even if intended machine has a geometry engine processor.

10 64 Chapter 3. FreeWalk: A Three-Dimensional Meeting-Place Scanning Area (a) Rotated Texture Image (b) Screen Image Figure 3.8: Drawing a 3D Space Since one of the standard devices of workstations is a mouse, the motion of users' pyramids is controlled by the X and Y valuators of the mouse pointer while its left button is down. The orientation of the pyramid is controlled by the mouse so that the moving/turning speed is proportional to the distance between the mouse pointer and the center of the FreeWalk window. Since users can easily control the speed of moving/turning, they can run when the target is in the distance and slow down as it becomes closer Grasping Situations in the 3D Virtual Space Since the viewing angle of the CRTismuchnarrower than that of human eyes, it is hard to grasp the surrounding situation. FreeWalk has special functions to solve this problem. Ahuman in the real world can easily look around by turning his/her head, which is often hard to do in the 3D virtual space. Virtual reality systems can simulate this by using a head-mounted display (HMD), which widely used machines are not equipped with. In videogames, on the other hand, additional functions such as auxiliary indications and view point switching are introduced to help users to grasp their situations. From this observation, we implemented the following functions in FreeWalk. Radar screen Participants can nd the locations of other participants by referring to a radar screen that occupies the right corner of the window. The radar screen indicates a simplied view of the surroundings, including the location and orientation of other participants. The radar screen

11 3.3. System Design of FreeWalk 65 FreeWalk window Radar screen Picture Name of participant Location of other participant Figure 3.9: Radar Screen Yourself (a) Normal View (b) Bird's-eye View Figure 3.10: Switched View Point can also indicate the volume of people's voices so that the user can roughly know the activities in groups. To identify each user, his/her name is displayed above the pyramid. The default names of the users are their login names, but they can specify their names before they start FreeWalk. Figure 3.9 shows the design of the FreeWalk window containing the radar screen. View point switching In some cases, the bird's-eye view is more suitable for grasping the situation. The view point switching function allows users to select an appropriate view from multiple viewpoints. Figure 3.10(b) shows the bird's-eye view from the location of the user's character. This view enables the user to watch both the user's character and his/her sur-

12 66 Chapter 3. FreeWalk: A Three-Dimensional Meeting-Place Community server Vision process Voice process Client Vision process Voice process Client Vision process Voice process Client Information of each client Information of all clients Data transfer Figure 3.11: FreeWalk System Conguration roundings. As a result, the user can have a better view of the geometric relations among participants, and thus move easily than using the normal view System Conguration The FreeWalk system consists of a community server and clients, each of which includes vision and voice processes. Figure 3.11 illustrates the interaction among the community server and clients. When a participant makes a move using his/her mouse, the corresponding client calculates the new location and orientation, and sends them to the community server. The community server then compiles the information from all clients into a list of IP address of clients with their locations in the 3D community common. The server nally sends the list back to each client for screen updating. Since only control information is transferred between the server and clients, the community server can eciently maintain a global view of the ongoing activities in the community common. When a client receives a list of the other clients, the vision process of the client sends its owner's picture to the other clients on the list. On receiving pictures from other clients, the vision process redraws the display based on the information in the list and the pictures received. Because not all clients can be seen by each client, it is not necessary for each client to send its picture to all others. Similarly, each client does not have to send full-size pictures to clients far away. FreeWalk uses these facts to optimize the bandwidth of video communication as follows. The sender adjusts the size of the picture to the size needed by the receiver.

13 3.3. System Design of FreeWalk 67 No pictures are sent between clients who cannot see each other. Figure 3.12 shows an example of video transfer in FreeWalk. Since client A is located near client C, a large picture is sent fromclient C. In contrast, client C sends a small picture to client B, which is located far away. B s view of C B A s view of C A C Picture size Large Small Figure 3.12: Video Transfer To further reduce the communication bandwidth, FreeWalk clients send pictures in an interlaced fashion. Neighboring vertical lines are sent by turn, rather than sending a full picture. Voice communication is performed in a similar fashion. FreeWalk clients do not send voice data to those clients that are too far away to hear the voice. The volume of voice is determined by the receiver, since voice volume does not contribute to bandwidth reduction. The two major problems with voice communication are voice discontinuity and delay. Playing the voice packets immediately upon receipt causes discontinuous voice since the arrival rate of each packet usually varies. To avoid this, voice packets are rst buered, and played as one sequence. This buering method, however, causes some delay in voice transfer. To reduce this delay, if voice playback lags the arrival of voice packets, a small part of these packets is discarded. The FreeWalk window isdrawn in real time. The rate of live video communication and the size of window can be specied before running FreeWalk. In addition, the size of window might also be changed while FreeWalk is in operation. When the window size and live video size is small, the time

14 68 Chapter 3. FreeWalk: A Three-Dimensional Meeting-Place consumed to draw the window is also small. This saves network and CPU resources. 3.4 Performance Evaluation We conducted experiments in laboratories, an intranet and the Internet to evaluate the performance of FreeWalk Experiments in Laboratories We organized six clients in dierent rooms of our department and validated our implementation policy. Although we only used a 10 Mbps Ethernet for transferring data, the six participants could meet and talk naturally. Figure 3.13 shows the situation. In this experiment, a SUN workstation and six SGI (Silicon Graphics, Inc) Indy workstations were used as the FreeWalk's server and clients, respectively. The major results we have obtained are as follows. (a) Single Group (b) Several Groups Figure 3.13: Six Participants in FreeWalk Each participant could move according to his/her own will. The six people formed several groups from time to time. People reported that they could share the same space without confusion. Various behaviors have been noted so far, such as approaching a pair of participants talking to each other from a distance to secretly listen to their conversation, chasing a moving participant while calling him/her to stop, and a female researcher who prefers to use her doll as a character for herself. Most of the participants enjoyed the experience due in part to its relaxed atmosphere.

15 3.4. Performance Evaluation 69 The interruption and delay of voice did not signicantly damage communication quality. Voice volume does not depend on sender direction, so when the receiver has two possible senders located at the same distance from the receiver, the receiver has some diculty in distinguishing who is speaking. In the current version, we use stereo sound to improve voice localization. The operation of FreeWalk is simple and most people are satised with its performance. There were a few people who found it dicult to control the moving speed using the distance between the mouse pointer and the center of the FreeWalk window. For this type of user, we added a mode where only the direction of movement isspeciedbythe keyboard while the speed is set constant. Since the radar view covers a wide area, it is not easy to distinguish adjacent participants. We are planning to make the range variable and customizable by users. Various impressions have been reported from the users who used FreeWalk experimentally. Most users felt that its user-interface is similar to videogames and as enjoyable as if they were playing a videogame. They also reported that the operation of FreeWalk is intuitively understandable, and easier than other desktop conferencing systems Intranet Experiments We rst installed FreeWalk on 50 SGI Indy workstations connected by a 10 Mbps Ethernet in Tohwa University. Though the initial test was successful, an over-concentration of clients caused network congestion and audio howling when about twenty participants joined at the same time. We then created an intranet meeting with FreeWalk in the event called Open Campus (the campus was open to the public) held in Tohwa University. The visitors of the event joined FreeWalk meetings without any scheduling beforehand. The meeting continued for about six hours and a maximum of 13 users participated simultaneously. As a result of investigating the log data, the following interesting user behavior in the 3D virtual space was found. Most people move around the center landmark of the space All participants did not try to go far from the center. As population density around the center became high, network trac exploded. This was because multimedia data of many participants were transferred to each client though he/she did not talk to most of them. A group of people moved together It was often observed that a couple of people moved together to a long

16 70 Chapter 3. FreeWalk: A Three-Dimensional Meeting-Place distance, but seldom more than three. Some users reported that they wanted to ride a bus, because it was hard to move together. Some people wandered from a group to group In the latter half of the meeting, the number of participants who moved around decreased. Moving participants then wandered from group to group. It was very often observed that a couple of people faced each other Internet Experiments In this experiment, four users at Kyoto University in Japan and one user at the University of Michigan in the United States joined the community server of FreeWalk at Kyoto University. The frame rate of 3D drawing was about 10 frames per second, the same as in the previous intranet experiments. Though the delay of the live video was larger than that of the intranet, it was inconspicuous and did not aect the control of the player's character much. The users reported they could hold a meeting that was as good as of the intranet. Sometimes the bandwidth between Japan and the United States forced us to lower the video frame rate to four frames per second. However, the users were still able to nd others smiling through the live video. The delay of audio was inconspicuous, too. However, the audio of the user in the United States was sometimes intermittent, and the other users were unable to catch what he said while he could clearly hear the voice from Japan. Further experiments found that the loss of UDP (Internet User Datagram Protocol) packets transmitting audio data caused the intermittent audio. 3.5 Interaction Analysis Comparison of Communication Environments Some earlier studies tried to compare the communication aided by conventional desktop conferencing systems (conventional video communication) with face-to-face communication (FTF communication). Various characteristics of conventional video communication became clear through those studies [Cohen 1982, Fish et al. 1990]. However, the characteristics of the communication aided by a desktop conferencing system with a 3D virtual space (3D communication) remained unclear. Our aim in this section is to show the characteristics of 3D communication compared with FTF and conventional video communications. We rst compare the conventional video and the 3D communication environments as follows.

17 3.5. Interaction Analysis 71 Participants using a conventional desktop video conferencing system (conventional video environment) tend to be strained and their conversations do not proceed smoothly. This is because all their faces are always displayed and the system keeps everyone facing the others. A 3D virtual space eliminates this strain by giving them locations and view directions. It is impossible to reproduce communication with moves like real life communication in a conventional video environment. A 3D virtual space reproduces communication with moves by enabling participants to move freely. We tookinperson 3 [SGI 1995], as the conventional video environment, and used FreeWalk as a desktop conferencing system with a 3D virtual space (3D environment). Table 3.1 shows the functional dierences between these two environments. Table 3.1: Functional Comparison Process of joining Maximum number of participants FreeWalk Enter a 3D virtual space voluntarily Unspecied (practically, 20orso) InPerson Called by someone who has already joined 7 Occurrence of conversation Caused by participants' approach of their own accord Meeting group Multiple groups Single group Caused by turning on the system by a coordinator Process of joining In FreeWalk, the process used to join a meeting is just to enter the 3D virtual space provided. Each user selects which virtual space to enter when he/she starts up the system. The conversation protocol of InPerson inherits that of telephones: in order to hold a meeting between two persons, one should call the other via InPerson. If one wants to join the meeting, he/she needs to be called by someone who has already joined it. A newcomer cannot join an InPerson meeting freely. 3 InPerson is a trademark of Silicon Graphics, Inc.

18 72 Chapter 3. FreeWalk: A Three-Dimensional Meeting-Place Maximum number of participants The maximum number of participants is seven in InPerson. This limitation is from the size of workstation displays. On the other hand, FreeWalk does not limit the number of participants, though if the number exceeds 20, the performance of the system becomes intolerable given the current condition of computer networks. Occurrence ofconversation In FreeWalk, a conversation may be started by an accidental encounter while the participants are walking around the 3D virtual space. A conversation is started by participants' contact of their own accord. In the case of InPerson, conversation is started when the coordinator of a meeting turns on the system and contacts all participants. Meeting group In FreeWalk, participants approach one another to organize a meeting group. Participants can form multiple meeting groups simultaneously. In the case of InPerson, however, participants always form a single meeting group since everyone faces the others and hears the voices of the others Hypotheses on Conversation Environments There are various studies that compare computer-mediated communication with FTF communication. Some of them are listed below. Fish observed communication among 23 people using CRUISER, a desktop conferencing system for three weeks [Fish et al. 1992]. CRUISER has functions that support informal communication. One of them is called Glance, which is used to observe other people readily, and another is Autocruise, which simulates accidental encounters as explained in Section 3.1. The result of the experiment was that CRUISER was used like telephones, that is, conversation topics were mainly greetings and schedule arrangements, and it was less used than FTF communication to make decisions or to solve problems. Mc- Daniel also compared FTF conversation with a computer-mediated system [McDaniel et al. 1996]. The result was that there were more extensive multiple threads in computer-mediated conversation, but both conversations were similar in content and nature of participation. Issacs studied how a distributed presentation over a computer system called Forum diered from FTF presentation [Isaacs et al. 1995]. The result was that audiences were larger but the quality of interaction was lower over Forum; audiences preferred to watch talks over Forum but speakers preferred to give talks in an FTF setting. Bowers investigated how avatars' moves were coordinated with

19 3.5. Interaction Analysis 73 conversation in a virtual environment [Bowers et al. 1996]. Results showed that avatars' moves were used for transferring the initiative of conversation. Sellen compared communication in two video conferencing systems, Hydra and PIP, andtheftf environment [Sellen 1992]. Hydra used multiple cameras, monitors, and loudspeakers in order to support directional gaze cues and selective listening. PIP used a single camera, monitor, and speaker, and a picture-in-picture device to display multiple people on one screen. No differences were found among the three environments for conversation in terms of turns (transferring the initiative of speech) and the amount of utterances. However, FTF conversations contained more simultaneous utterances than either desktop conferencing environments. We expect that 3D communication could be more similar to FTF communication than conventional video communication. We thussetupthe hypotheses described below according to Sellen's experiment in terms of the patterns of conversation, that is, the amount of utterances, the number of turns, and the amount of simultaneous utterances. H1 There is no dierence between conversation environments in terms of the amount of utterances. H2 There is no dierence between conversation environments in terms of the number of turns. H3 The amount of simultaneous utterances is the most in the FTF environment and the least in a conventional video environment. We set up the following additional hypothesis since no evaluation results have been published on communication in a 3D environment. H4 Participants' moves in FTF and 3D environments are similar Experimental Setup Design of experiment All people who participated in the experiment were university students. They consisted of eighteen males and three females. Three environments for conversation were prepared to compare three dierent communications: FTF, conventional video, and 3D communications. Figure 3.14 shows the three environments. 1. Face-to-face environment (FTF) Participants had discussions in a small conference room (25 m 2 ). People were requested to keep standing so that they could move during the discussions.

20 74 Chapter 3. FreeWalk: A Three-Dimensional Meeting-Place (a) FTF (b) InPerson (c) FreeWalk Figure 3.14: Three Dierent Environments for Conversation 2. Conventional video environment (InPerson) Seven SGI O2 workstations connected by a 100 Mbps Ethernet were prepared, and the InPerson meeting environment was set up. Four of the workstations were set in one room, carefully placed so that the participants could not see each other directly, and the remaining three were placed in three separate rooms. Each workstation was equipped with a headphone to prevent users from hearing voices outside the system. The pictures of all the participants were displayed in the screen of each user, arranged in three rows as depicted in Figure 3.14(b). The voices of the participants could be heard simultaneously. 3. 3D environment (FreeWalk) FreeWalk was set up on the same workstations as InPerson. Each participant could hear his/her own voice but could not see his/her own picture. The meetings held in the three environments consisted of three tasks: agreement for the destination of group travel, discussion about social problems, and free conversation. These tasks were selected in order to examine various types of communication comprehensively. We did not choose any

21 3.5. Interaction Analysis 75 chairmen of the meetings in advance. The period of each meeting was 20 minutes. For each task, participants were told to organize three groups. One group consisted of six males and one female. Each task was assigned to each group in a dierent environment. Tasks are described precisely as follows. 1. Agreement for the destination of group travel (Task1) This was a decision-making task. We made the participants decide where they would travel a month later. They were asked to pretend to be friends in high school days who did not have many chances to meet after they left university. We selected group travel as the topic so that they could talk easily. 2. Discussion about social problems (Task2) This task was to shape ideas. They were asked to pretend that they attended the same lecture and had to hand in reports. There were six keywords, namely, aging society, the consumption tax system, the annuity system, the lifelong employment system, the seniority system, and the information-oriented society. We selected these as the subjects of discussion, and a written report was required so that they could feel this task was realistic. 3. Free conversation (Task3) Participants had conversation without any guidelines. For a summary, we prepared three environments and three tasks, so nine types of meetings took place. Table 3.2 shows the combinations of tasks and groups. The whole experiment was performed on the same day and progressed as follows. Before performing the three tasks, people who participated in the experiment introduced themselves in each group so that they could memorize their faces and their voices respectively. They also practiced the operation of FreeWalk for ve minutes before the experiment. After the experiment, they were requested to ll out questionnaires after they accomplished all three tasks. Table 3.2: Combination of Tasks and Groups Task1 Task2 Task3 Group1 FTF FreeWalk InPerson Group2 InPerson FTF FreeWalk Group3 FreeWalk InPerson FTF

22 76 Chapter 3. FreeWalk: A Three-Dimensional Meeting-Place Data collection We collected experimental data using video tape recording and questionnaires. In addition, we analyzed the system logs of FreeWalk to nd the pattern of moves in the 3D virtual space during meetings. Video taperecording During FreeWalk and InPerson meetings, the screen images of the workstations were recorded on video tape recorders. In FTF meetings, the scenes were recorded on 8 mm video. The analysis was done as follows. First, we reviewed the video tape pictures to record the start and end times of participants' utterances to create conversation records. From this review, we classied utterances into two categories: reactive utterances and normal utterances. The former is a reaction to an utterance of another person, such as giving a response or a laugh, and the latter is all utterances except reactive ones. Next, we calculated the amount of utterances, the standard deviation of utterance, the amount of simultaneous utterances, and the number of turns from the conversation records. Finally, we examined the relations between these values and various factors, including groups, tasks, and environments. Questionnaires Participants lled out the following four questionnaires in free format. 1. About FreeWalk 2. About InPerson 3. Most favorite task 4. Most uninteresting task System logs In FreeWalk, locations of participants in a 3D virtual space were recorded. We analyzed these system logs by plotting the locations. Tools for analyses We developed two tools to analyze the experimental data. One is SimWalk, which we used to analyze participants' moves in a 3D virtual space. SimWalk visualizes the records of moves in the 3D virtual space. The other is SimTalk, which visualizes conversation records to analyze their patterns. The functions of both tools are described as follows. 1. SimWalk The FreeWalk community server stores system logs, in which the following data of each participant were recorded at each second:

23 3.5. Interaction Analysis 77 IP address of each client machine; name of each participant; location of each participant in a 3D virtual space; orientation of each participant in a 3D virtual space; volume of voice picked up by microphone. SimWalk can reproduce participants' moves and plot their locations using the data in system logs. Figure 3.15 is a screen image of SimWalk, which is reproducing participants' moves. Figure 3.15: Screen Image of SimWalk Reproduction of moves SimWalk draws triangles that represent the participants on a screen. The locations and orientations of these triangles correspond to those of the participants in a 3D virtual space. The triangles blink to indicate utterances if voice volume is greater than a xed value. The triangles move as the system clock progresses. Including normal replay, SimWalk also has various modes like a VTR: for example, reverse play, pause, and play frameby frame. While SimWalk is reproducing the moves, we can change the parameters to customize SimWalk, that is, replay speed, size of screen, and so on. Plotting locations SimWalk can plot participants' locations by reproducing the moves, and can draw lines along each participant's moves by connecting his/her locations in sequence.

24 78 Chapter 3. FreeWalk: A Three-Dimensional Meeting-Place 2. SimTalk SimTalk visualizes the pattern of conversation in various drawing modes and calculates values from the conversation records. In the graph, the vertical axis represents time and each row corresponds to each participant. A record consists of the following data. name of the speaker of an utterance; start time of utterance; end time of utterance; type of utterance. We can select the drawing mode that ts each subject, that is, turns, simultaneous utterances, and so on. Flows of utterances Each utterance is represented by a black bar. The vertical axis represents time. Amount of simultaneous utterances The bars are colored according to the number of simultaneous utterances. Turns The vertical axis represents time. The graph is marked according to the time when each turn occured. SimTalk can also calculate various factors such as the amount of utterances, the amount of simultaneous utterances, and so on Results Conversation We analyzed the pattern of conversation and the occurrence of chat. For the analysis of the pattern of conversation, we retrieved a two-minute period from the conversation records. Figure 3.16 shows the ows of utterances drawn by SimTalk (Group 2). In these charts, each row corresponds to one participant. It can be seen that utterances in Task2 are more sparse than those in Task1 and Task3. To analyze the pattern of conversation, we used SimTalk to derive four target values, as follows. 1. Amount of utterances is the total time of all utterances of participants.

25 3.5. Interaction Analysis 79 (a) Task1 (InPerson) (b) Task2 (FTF) (c) Task3 (FreeWalk) Figure 3.16: Flows of Utterances (Group 2) 2. Standard deviation of utterance is the standard deviation of the ratio of the total time of utterances of each participant to the total time of all utterances of participants. 3. Number of turns is the number of events, each of which transfers the initiative of talking from one to another. The turn occurs when someone starts talking immediately after or while another talks. The case where someone stops talking and starts talking again after a silence is not counted. 4. Amount of simultaneous utterances is the sum of time periods when multiple participants talk simultaneously. 5. Occurrence of chat is an event of starting a conversation that does not contribute to the task. Suppose one wants to accomplish the task; then he/she should propose the destination of group travel and the mean of transportation as an example for Task1, and for Task2, he/she should answer what the keywords mean and how they relate to one another. Therefore we regard the series of utterances that do not contribute to the direct accomplishment of the task as the occurrence of chat. The analysis results of the conversations according to the above denitions are described below.

26 80 Chapter 3. FreeWalk: A Three-Dimensional Meeting-Place (sec) 100 Group1 Group2 Group3 (number of turns) 100 Group1 Group2 Group3 (sec) 15 Group1 Group2 Group (a) Task1 Task2 Task3 Amount of utterances Task1 Task2 Task3 (b) Number of turns (c) Task1 Task2 Task3 Amount of simultaneous utterances Figure 3.17: Measured Values for Each Task 1. Amount of utterances Figure 3.17 presents bar charts that show the amount of utterances, the number of turns, and the amount of simultaneous utterances. Figure 3.17(a) shows the signicant dierence of the amount of utterances across groups. The amount is largest in Group3 and smallest in Group1, while no dierence is observed across tasks. The amount of utterances is aected by the dierence of groups, not by tasks or environments. The ranking of the contribution of groups to the amount ofutterances is as follows: Group3 > Group2 > Group1 2. Standard deviation of utterance Table 3.3 summarizes the standard deviations of utterance. It arranges the values by task and environment. The following ranking of environments for each task were obtained from the table. Task1 FTF > InPerson > FreeWalk Task2 FTF > InPerson FreeWalk Task3 InPerson > FTF FreeWalk

27 3.5. Interaction Analysis 81 Table 3.3: Standard Deviation of Utterance Task1 Task2 Task3 FTF InPerson FreeWalk (number of turns) 100 FreeWalk InPerson FTF (number of turns/ amount of utterances) 1.2 FreeWalk InPerson FTF Task1 Task2 Task3 (a) Number of turns Task1 Task2 Task3 (b) Frequency of turns Figure 3.18: Number of Turns The standard deviation of utterance is aected by tasks and environments. The interesting fact is that the deviation is the smallest in FreeWalk for all tasks. This means that the amount of utterances of each participant is equalized in FreeWalk. 3. Number of turns Figure 3.18 includes bar charts that show the number of turns and the frequency of turns. The frequency of turns is the number of turns divided by the amount of utterances. Figure 3.17(b) shows that the number of turns for Task2 is the least in all the tasks. Figure 3.18(b) shows the clear relation between the frequency of turns and environments. Thus, the number of turns is aected by the dierences in tasks and environments. The rankings of contributions of tasks and environments to the number of turns are as follows: Task1 Task3 > Task2

28 82 Chapter 3. FreeWalk: A Three-Dimensional Meeting-Place FreeWalk > FTF InPerson The number of turns is small in Task2, since participants mainly posed problems and asserted themselves, but it is large in Task1 and Task3, where short exchanges such as questions and answers predominated. The eect of the dierence in environments shows FreeWalk activated turns more than InPerson and FTF. 4. Amount of simultaneous utterances Figure 3.17(c) shows that the amount ofsimultaneous utterances is less in Task2 than in Task1 and Task3. It also shows that the amount is the most in Group3 and the least in Group1. Thus, the amount of simultaneous utterances is aected by the dierences in tasks and groups. The rankings of the contributions of tasks and groups to the amount ofsimultaneous utterances are as follows: Task1 Task3 > Task2 Group3 > Group2 > Group1 The eects of the dierences in tasks on the amount of simultaneous utterances are similar to the case of the number of turns. Readers may recognize that Group2's Task3 meeting has a large amount of simultaneous utterances, but this is an indirect eect of the dierence in environments. The environment of Group2's Task3 is FreeWalk and participants' moves are signicantly dierent from other FreeWalk meetings: participants formed a disorderly pattern to have a conversation in Task3. Details about their moves will be described in the next subsubsection. 5. Occurrence of chat Figure 3.19 shows the occurrence of chat in Task1 and Task2 in each environment. In this gure, the horizontal axis represents time and each mark represents the occurrence of chat. In this gure, it can be seen that chat occurred more actively in Task1 than in Task2, and more in FTF than in FreeWalk, while it seldom occured in InPerson. The occurrence of chat reects the dierences in tasks and environments. The rankings of the contributions of tasks and environments to the occurrence of chat are as follows: Task1 > Task2 FTF > FreeWalk > InPerson

29 3.5. Interaction Analysis 83 Task1 Task2 FTF InPerson FreeWalk (min) Figure 3.19: Occurrence of Chat In FreeWalk, the atmosphere among participants might have been relaxed since they formed a circle to have a conversation, while everyone faced the others in InPerson. Participants' moves In FTF meetings, participants seldom moved after forming a circle to have a conversation. In InPerson meetings, everyone faced the others on the screen. Figure 3.20 shows participants' moves in FreeWalk meetings. SimWalk plotted their moves during a period of fteen minutes. In Task1 and Task2, they seldom moved after forming a circle as in FTF. Unlike the other two tasks, they moved actively around the 3D virtual space in Task3, as shown in Figure 3.20(c). This means that the 3D virtual space activated the participants' moves. In Task3, which is free conversation, the following behaviors were seen: 1. Enjoy moving in a 3D virtual space At the beginning of the task, participants moved actively. For example, they moved to the edge of the 3D virtual space and rushed towards others. The occurrence of conversation was scarce. 2. Face each other to greet In the middle of the task, participants faced one another frequently to greet. The lengths of conversations were very short. We noted that some participants blamed others for approaching them when they tried to whisper to each other.

30 84 Chapter 3. FreeWalk: A Three-Dimensional Meeting-Place (a) Task1 (b) Task2 (c) Task3 Figure 3.20: Pattern of Moves in a Three-Dimensional Virtual Space 3. Gather to start conversation Towards the end of the task, all participants gathered to converse. They did not move actively after conversation started. We noted that a certain participant ran about trying to escape from the meeting place since he was unwilling to talk, while another participant looked for someone else who had gone elsewhere. People often talk while they walk in real life; for example, they walk side by side to talk to each other. In the FreeWalk meeting in Task3, participants moved to communicate, but they seldom talked while they moved. The reasons for this phenomenon are as follows. Participants did not have to talk while they moved in order to accomplish the tasks. Moving in a 3D virtual space by controlling a mouse has to be done consciously, but a walk in real life is usually done unconsciously. Participants could not ascertain that they were actually moving together since the eld of view of a 3D virtual space in the FreeWalk window was too narrow to watch the other participant who stood side by side. Questionnaires Table 3.4 shows participants' answers to the question \most favorite task" classied by groups and environments. \G1," \G2," and \G3" mean Group1, Group2, and Group3, respectively. The members of Group1 accomplished Task1 in FTF, Task2 in FreeWalk, and Task3 in InPerson (see Table 3.2). Two members of Group1 selected Task1 as the most favorite task. This table