HyperMirror: Toward Pleasant-to-use Video Mediated Communication System

Transcription

1 HyperMirror: Toward Pleasant-to-use Video Mediated Communication System Osamu Morikawa National Institute of Bioscience and Human-Technology, 1-1 Higashi, Tsukuba, Ibaragi ,Japan ABSTRACT We designed HyperMirror to provide a new video image that presents an attractive, highly understandable communication environment, rather than imitating face-toface communication. The HyperMirror environment enables all participants to feel they are sharing the same virtual space. Participants communicate using images meeting the condition "What I See Is What You See" (WISIWYS). Both local and remote participants appear together on a shared video wall, and all things on the wall - - even those out of reach -- become appear to come within reach. Participants sharing the screen tend to act as if they are in the same room. Keywords HyperMirror, WISIWYS, mirror image, awareness, video conference, telepresence, interpersonal communication INTRODUCTION Communication can be roughly divided into two cases -- one emphasizing information transfer/exchange and one focusing on communication space sharing. Information transfer/exchange has been mainstream in attempts to develop or improve existing videophones and video conferencing systems. For such information systems to become widely disseminated in the general population, however, it is also important to consider the sharing of a communication space -- an environment in which people can enjoy conversation itself with friends and family. The telephone, for example, was originally developed to transfer/exchange information, but its subsequent, widespread population was due primarily to the recognition that it also could bring people together to share communication space. Despite attempts at sophistication and popularization, videophones and video conferencing systems are being used Takanori Maesako Faculty of Human Sciences, Osaka University, 1-2 Yamadaoka, Suita, Osaka , Japan maesako@mcgyver.hus.osaka-u.ac.jp in only a few limited applications. This suggests problems with the technology that prevent it from being accepted more widely -- in sharp contrast to the immediate, explosive spread of the cell phones and portable phones that are the second-generation in telephone applications. Our purpose is to clarify this problem with videophones and video conferencing systems in communication space sharing and to suggest a possible solution. WHY DID PEOPLE EMBRACE THE TELEPHONE SO ENTHUSIASTICALLY? A study by Yoshida et al. [28] showed that college students in and around Kyoto-Osaka (effective response: 549) cited the following reasons for the phone's popularity : speed, available anytime, available anywhere, no visual information about the other end, and easier way to say what we want than in face-to-face conversation. They also cited the following disadvantages: no visual information about the other end and difficulty in conveying subtle emotion. How interesting that no visual information about the other end becomes both an advantage and disadvantage! The above study suggests that the telephone provides its own conversation environment rather than a substitute for face-to-face conversation -- in short, people have found a new way to communicate using voice alone. Why has this proven to be so appealing? Conveying voice alone, the telephone does not let people share a physical world --individuals at either end share a mutual, voice-only world in which the same sound information is provided to both speaker and listener(fig.1). Voice alone limits other forms of information, but is complete enough to make this sound-only world acceptable sound world sound world Phisical world Phisical world Visual world Fig.1 The sound world by telephone is shared. So the sound information is symmetrical. Visual information is not symmetrical at a Videophones

2 Face-to-face -- and very widely so. Video-mediated A B C A B C A B C Fig.2 "Mona Lisa effect" by video medium PROBLEMS WITH CONVERSATIONAL VIDEO ENVIRONMENT The video conversation environment provides other forms of information -- images -- in addition telephone functions. If possible, most people would probably like to see as well as hear people at the other end. Displaying faces on a screen, however, raises a new problem. Despite its apparently familiar presentation of a face, but it also conveys dissonant information, for example, about gaze direction. "Gaze Direction" Gaze direction, important to communication, serves at least five functions[10] -- regulating the conversation flow, providing feedback on how communication is perceived by the viewer, communicating facially evident emotion, communicating the nature of the interpersonal relationship, and preventing the input of excess information. The video conversation system determines the speaker's image by the position relative to the camera. This image is interpreted much the same way by most people, independent of position -- a type of Mona Lisa effect in which a person passing in front of the famous face feels that those eyes are following them[1][17][19](fig.2). It is not possible to direct the speaker's gaze to only one person with more than one listener present -- in other words, the speaker's gaze cannot be interpreted by all listeners as "face-to-face". As Gaver[3] and Sellen[22] observed using a simple video conversation system having both video and audio channels, information on the gaze direction could not be used as effectively in video conversation, as in face-to-face conversation. Simultaneously, however, at least a few participants understood, based on the situation, that the speaker intended to talk "to them" even though the gaze was not always face-to-face, or, in other words, did not always allow direct, mutual eye contact between speaker and listener. Dourish et. al [2] observed similar phenomena. "Looking into the camera" provides feedback to the speaker that the listener is, indeed, paying attention. To develop a greater awareness of mutual gaze patterns, however, the speaker must learn to recognize when the listener's gaze is directed at the video monitor and, hence, that the listener is paying attention. The desired understanding of "eye contact" is replaced by the understanding of "being looked at". Gestures Communication does not, however, limit itself to facial expression, of course. Gestures are highly important, for example. This understanding has lead researchers to enlarge displayed area to include gestures. A study by Heath [6][7], showed that it becomes harder to understand gestures transmitted by video, even though the viewer is clearly communicating a readiness to listen or an indication of understanding. To compensate, people tended to exaggerate their gestures, which made things worse and did not add to the progress of conversation. Their study even reported a case in which gestures indicating a readiness to start conversation were not understood even when watching the screen. We do not know yet why gestures are not conveyed smoothly. It may be because gestures are understood as interaction to be interpreted together with environmental information, rather than as something to be directly interpreted for its own meaning alone (Fig. 3). Pointing In current video conversation systems, we can point to objects in our own surroundings, but not to those in the viewer's space. This prevents such common gestures related to the relative positions of the speaker and listener as "come closer" or "move farther away". It also prevents "touch" gestures of physical interaction, such as patting the other person's shoulder or shaking hands. Common Resources In face-to-face conversation, we share objects in our surroundings -- like reading someone's note and appending additional information. With video as it is now, a speaker can show the viewer a note, but not to see a note from the viewer or write anything on that note Fixed Viewpoint In face-to-face conversation, we watch something interesting by turning toward it, approach closer, perhaps move it somewhere else. Video conversation viewers have? Fig.3 Disregarded exaggerated gestures at videophones.

3 no such freedom. The communicator also faces restrictions limited camera-view display area, no freedom of movement. IMPROVED VIDEO CONVERSATION SYSTEMS These problems are, in short, caused by the wide gap between the spontaneous desires of users when video became available and the functions actually provided by the video conversation system. Since then, many reports have reviewed these lacks and proposed solutions. To support multiple eye contact, Sellen et al.[22] developed a system using a Hydra Unit in which a camera, display, microphone, and speaker are integrated compactly. The display is small and the camera is located just above it, almost maintaining eye contact. The small display makes the speaker look smaller, removing the feeling that life-size images give. To make participants feel more like they are sharing the environment, Okada et al.[21] developed MAJIC life-size display. Tang's VideoDraw [25] stresses the importance of common resources in communication. Limiting communication to collaborative drawing, the system displays users' faces and the related drawing image--a canvas and users' hands. This makes pointing and other gestures meaningful. Tang's translucent VideoWhiteBoard [26] displays silhouettes of the whiteboard and those writing on it, sacrificing face display but enabling users to recognize each other by silhouette. Ishii's ClearBoard [9], for another example, makes facial expressions observable using a transparent board. These methods still leave participants physically separated, fixed and able to work only near the screen. Their only sharable space is the two-dimensional canvas. Kuzuoka, et al.[12] helped realized gaze awareness using three-dimensional objects on a desk that serves as a common resource and creating images as if two people face each other across the desk. This is a special case of face-toface communication. Another approach to these problems creates a "virtual" video conversation environment rather than imitating faceto-face communication. In Ishii's TeamWorkStation[8], common communications resources are actual objects on a table. Images are displayed combined on the tables of all participants, together with their hands. Wellner's DigitalDesk[27] features a fusion of these table and hand images with computer graphics to create common communications of resources. These systems achieve communication by having participants manipulate objects and use gestures. With Bruce Blumberg's ALIVE system[14] and Sommerer's MIC Exploration Space[23], participants are not actually seen, but are displaced similar to "avatars", who use objects in the display, such as electronic pets and electronic plants. Participants view their own as computerized graphics as they communicate with objects in the computer by gesture. Kazuoka's GestureCom [13] gives the viewer camera control. This eliminates fixed viewing. Participants being imaged can determine the direction the camera is facing, enabling them to know what is being viewed. This produces a new gaze direction effect differing from the face image. Yet other systems focus on relative user position. "Virtual Self" icons,--a sort of "avatar"--for example, represent users in electronic virtual space. By manipulating icons with mice, etc., users communicate directly within the same virtual space, called InterSpace [24], MASSIVE [4], and GreenSpace [15]. The reality of the participant's presence in the same room, however, is very different from face-to-face communication because icons, --not users themselves--are located in the same space. Krueger's VideoPlace[11] displays the silhouettes of full torso or hands of the participants. Gestures are then used to communicate. As can be guessed, feeling of being in the same space as other participants is stronger than in previous examples. Because these are silhouettes, however, information and the like cannot be used interactively. Fig.4 An Example of HyperMirror conversation. HYPERMIRROR HyperMirror proposes a solution that enables all participants to be project into the same room--but not by imitating face-to-face communication. This system takes into account limitations-with telephones, the "voice-only" fact-and eliminates dissonance by making communication itself easy through careful delineation of the communication space. Even with existing video conversation systems, of course, the endlessly inventive human can create coping strategiessimulating eye contact by watching the camera and cuing return contact by watching a monitor--using gaze direction information in a way suitable for video. Unfortunately, existing video conversation systems are not the best way to do this because it takes too long to adjust to the "strange" communication environment. What we need is to design an

4 environment enabling new, easier or more natural strategies that do not distort other parts of daily interaction. Camera sensor Projector Fig.5 An example of HyperMirror system. Computer Better Understandability HyperMirror [16] enables participants to communicate using the same "video wall"--making it "What I See Is What You See" (WISIWYS). More than just two communications sites, can be set up with users in front of the system, eliminating headgear and other interfaces. Images are mirror reflections projecting users on a single screen--in the same room--regardless of physical location. HyperMirror image does not separate communicators and viewers: it puts them all in the same space.. This WISIWYS-ness lets HyperMirror users monitor, transmit, and create images as they like, making it easy to present objects, point to those belonging to others, and indicate things in shared space.(fig.6) HyperMirror Pointing In face-to-face communication, each user views others at a slightly different angle, preventing common physical image information from being identical, in the strictest sense, for all viewers. This difference is most easily understood by attempting to point to beyond-reach objects, such as mountains. Advantages of Reflected Image Display People hardly ever see themselves face to face except in mirrors. Reflected images are more familiar to most than nonreversed video--or "normal"--images. People move more naturally when they can see their reflection. To tap a person next to you on the shoulder, you move naturally when you can see your reflection because it is like what you see in the mirror. Psychological subjective evaluations related to embarrassment, affinity, etc., in showing people both reflections and normal images showed that most people were more comfortable with their reflections[18]. Creating a Relaxed Communication Environment In face-to-face communication, eye contact occurs when people sitting across a table from each other look straight ahead into each other's eyes--a normal environment--to achieve eye contact. In such a case, if there is no eye contact, it has been avoided intentionally, creating strain that may be a factor preventing people from enjoying conversation. Sometimes two people who are close sit at a 90-degree angle to relax and enjoy a conversation. They do not have straight-ahead eye contact. Eye contact is purely intentional--communication is relaxed because the lack of eye contact is not considered rude, yet it is easy to make eye contact when desired. In HyperMirror communication, people are side by side on a screen but look straight at each other. Due to difficulty in transmitting gaze direction information in video image, eye contact is not so intense when looking at others on a screen. As a substitute for face-to-face eye contact, you can turn to the person on the screen and move synchronously with them. This provides relaxed communication much as in the seated, 90-degree angle case (Fig.7) Fig.6 HyperMirror Pointing. videophone HyperMirror Fig.7 Relaxed conversation environment

5 A + B overlapping area C: optical Trick HyperMirror Image E: optical D: sensor F: electronical Fig.8 Variables of methods of making a HyperMirror image. HyperMirror requests to cover the upper half of the body, and to have overlapping area. HYPERMIRROR HARDWARE HyperMirror system implementation must consider parameters such as displayed body area, display size, and image resolution. If whole-screen resolution is fixed, enlarging the displayed body area results lowers resolution. Such critical parameters influence overall system performance. In a preliminary experiment, we found necessary to cover an area large enough to display fingers unless elbows are akimbo, although conventional Videophones cover only the face. Life-size display was preferable, although reduced display showed no problem in practical use. We also found that the displayed size, vertical position, and colors are more important than the accuracy of absolute display size. Basically, any participant can be displayed anyplace on the screen, although limiting movable areas showed no problem in practical use. We also found in the preliminary experiment that overlapping adjacent participants appropriately--shoulder to shoulder, at least--is necessary. Otherwise, participants are separated by invisible "window frame", destroying the HyperMirror advantage. We must then determine how to synthesize images--either electronically or optically. Optical synthesis usually uses double projection(fig.8-e and Fig.9). Some optical tricks also give display priorities to component images picked up at each end. Electronic synthesis enables optical tricks and sophisticated image processing in synthesizing images. Position sensors, for example, can be used to automatically Fig.9 A HyperMirror image of optical version. judge which component images should be displayed on the top of the screen where multiple component image overlapped. This enables a more natural image to synthesized [23]. EXPERIMENT Adding Self-Reflection to Displayed Participants HyperMirror display reduces the psychological distance between the viewer and other participants. A psychological test was conducted to confirm this effect [18]. In the experiment, three types of display conditions are set up: (a) no display of the viewer, (b) display of the viewer as in a mirror image or photo on the left side of the screen and as a normal, nonreversed image on the right, (c) the viewer on the left side of the screen and the mirror image on the right. The experiment lets participants subjectively evaluate where they think others are located or the screen in relation to them in communications and the degree of strength of the presence at that position. Subjects included 13 men and 12 women aged 10 to 50 years. The test was conducted in groups of three or four persons under the condition that the persons in the group Video image Mirror image only a partner Fig.10 Details of Experiment. 300cm

6 were strangers. Detail of Experiment A screen 90 cm high and 120 cm wide was installed upright 90 cm above the floor, and adjusted until participants were displayed life-sized at actual height. Projectors were placed 170 cm away from the screen and 90 cm above the floor. A camera was mounted on a projector. Subjects were 300 cm from the screen. The camera and screen had resolution compatible with NTSC television signals. Using two projectors, we created a synthesized image representing the subject and other participants in the same room (Fig.10). We set up two situations. Participants basically did not move much in one situation, but interacted by shaking hands, etc., in the other situation. Each was evaluated both with and without previously instructing the actual participant positions. This yielded 12 (3 x 2 x 2) cases. In each case, subjects were asked to fill in a form showing the location of each participant based on their feeling, marking 1 for right-front-of-screen, 4 for next right, etc. For "realism", the level of feeling was reported using 5 levels form, 5 for "extremely strong" to 1 for "weak." We told subjects it was not unusual to feel the presence of a participant in more than one place, such as both on the screen and to their right simultaneously, and asked them to mark what they really felt. After each experiment under different conditions, we also had subjects comment freely about comfort in talking, any noticed events, and any other impressions. The reply form also included a category such as "uncertain location" for subjects who felt a presence but could not determine where it was. Prediction Visual information, existing knowledge, and varied feedback were provided to recognize the position of other participants [20]. If the subject's reflection is not displayed, "screen data" is derived from visual information and "specified position" from knowledge about participant positions. Because it is usually abnormal for one person to be in two places at once, subjects try to solve this contradiction, but this action was stopped early on by our guidance, so it was predictable that two positions would be selected by many. If the subject's reflection is displayed, the presence of nearby participants is perceived because seeing the reflection provides feedback and images are displayed on the same screen. Interacting through the screen shows subject reactions immediately, again providing feedback for motion and the perception of the presence of others when an interaction succeeds. Perception also differs depending on which image--normal or reflected--is displayed. A subject recognizing the screen as a mirror is more likely to select "around me" with smaller displacement when reflections are displayed. When actual participant positions are not taught, no knowledge contradicts visual and feedback information, resulting in more subjects selecting "screen" or "around me", as responses. (Fig.11) Results Our results were almost as predicted. Reality of presence was evaluated highest at the screen itself, followed by specified actual positions. We calculated the average reality of presence with an evaluation 0 set to nonselected position (presence not felt). Results indicate that displaying a subject's reflection raises the reality of presence nearby except for "actual position not taught and interaction instructed", in which case a significant difference was seen (Student's t-test with a significance level of 5% or lower). In "actual position taught and interaction instructed" combined with the subject's reflection, reality of presence was highest at the right of the subject, higher than in specified actual position and on the screen When a normal image was displayed, reality of presence nearby was low and toward right on left, indicating that normal images did not effectively reduce the reality of presence in specified actual position or on the screen (Fig.12). Video image S L knowledge Mirror image specified mirror right image position Output S only partner's image pertner's image + feedback knowledge R video image partner's image left screen surface Garbage video image + knowledge + feedback miiror image + knowledge + feedback Fig.11 Prediction of experiment. There is a partner on the right of me in the screen, and s/he is on the side of raising my hand. (Video image condition)

7 score 3 blue wall room A A from camera B Screen Specified Position Left Right Fig.12 Results of experiment miiror image + knowledge + feedback video image + knowledge + feedback pertner's image + knowledge only partner's image 300cm room B A mixer by chromakey B to projector Fig.13 the HyperMirror system used observation HyperMirror Image Under "comments", one subject said "handshaking was easy in some cases and very difficult in others." We explained the difference between reflected and normal images and requested a handshake retry, after which the subject said "Handshaking is easy and my hand extends unintentionally when the reflection is displayed"--probably using a mirror-image metaphor in recognition. OBSERVATIONS ON SYSTEM USE The observation of communications using HyperMirror led to some interesting behavior. HyperMirror Hardware Used Observation As mentioned in the discussion of HyperMirror hardware, the HyperMirror system offers many possible variations on how the combined image is created. The HyperMirror configuration for observation is shown in Fig.13. Two conversation sites are used. The video signal is NTSC and the composite screen is generated using chromakey. For chromakey synthesis, the wall of one of the sites is blue. The sequence of screen synthesis of the two sites is predetermined, with the blue background taking priority. In screen overlapping display is always closer to the viewer, with no image hiding. Hardware uses only commercially available products-no special equipment or off-the-shelf products. Appearance of Socializing Due to Presence in Same Room Some subjects excused themselves, quickened their steps slightly, or stooped down while they passed in front of other participants and changed their positions on the screen. Such behavior was not observed when they passed behind others. Reflections displayed in a common HyperMirror communication image influence how subject act. They often grasped the image of other participants passively as though not actually relating or interacting in the case of a normal image. When a reflection was displayed, they respond at least to their own reflection. The HyperMirror environment gives no border between subjects and other participants, so they respond to the whole screen. It also affects how they recognize other participants and surrounding objects. The above behaviors appear to occur because a subject's reflection appears to interfere or interact with images of others. Similar images are provided by Krueger's VideoPlace[11], which also displays participants in silhouette. The line of vision is expressed by body positioning. Fig. 14 Examples of voluntary visual interactions. (Left) kicking man soon after shaking hands.(right)head patting man

8 Fig.15 An example of pointing. She will avoid from pointing man soon after this picture. Gaze Direction Expressed by Body With HyperMirror, gaze direction information accuracy of participants is low, since eye contact cannot be gained by looking straight ahead since the camera is located next to the screen. Nevertheless, users conversed smoothly, turning to the speaker on the screen and gazing at them in eye contact corresponding to face-to-face communication. In one case, a user tapped another person's shoulder on the screen in an attempt to start a conversation. Uniformity In many cases subjects talked to others through the HyperMirror image, not directly, even when physically in the same room. After such conversations, we asked them who was nearest to them during the conversation? Most people answered it was the person near them on the screen, not the person actually present in the same room. Thus, physical distance tends to be ignored. This is probably because HyperMirror creates a reality of presence exceeding that of physical presence. Personal Space Each person has "personal" space feeling that causes discomfort if invaded. This space depends on the relationship to the person violating this space[5]. We observed with HyperMirror that users tried to keep far enough away in order to avoid invading personal space-- evidence of the reality that users have in HyperMirror interaction. Approaching the Camera for a Close-up In a case details of the object in question could not be transferred to other participants due to low screen resolution, the person providing the information spontaneously help the object out toward the camera for a closer look. When doing so, they moved to keep from covering other people's faces. Other participants also Fig.16 Approaching the Camera for Close-up. moved to keep their faces visible. Everyone cooperated in creating an appropriate image for that moment (Fig.16). From" That" to "This" As mentioned, because HyperMirror uses WISIWYS, gestures are easier to communicate. Interestingly, the word subjects used to indicate objects, whether near or far, was "this" in all cases-meaning that objects seen on the screen had become close, at least psychologically, giving a sense of ability. In dealing with distant or three-dimensional objects, communication is usually tried after changing objects into pictures or names (symbols) that are more easily handled. However, in HyperMirror, the object is automatically imaged that all participants can touch it. Some users commented that pointing to objects in the air tires their hands and deprives them of actual "pointing" and that it is easier to point to objects by touching the blue background. HYPERMIRROR-II In the initial version of HyperMirror, all participants used reflection, so it was difficult to handle some objects, such as letters, maps, and drawings. To overcome this, HyperMirror-II displays the person as reflection and other participants as a normal image-although this is limited to two sites. Except that images at each end are the reverse of the other, the system satisfies WISIWYS. With the self-displayed in reverse, the same function can be expected as in the previous version. Letters, maps, etc., displayed as reflections may cause confusion, however. All objects displayed in reverse are present in real space where the user is. In other words, the user is able to obtain the normal images by looking at actual objects as required. In the previous HyperMirror version, an object pointed to on the HyperMirror screen was used in real space with no

9 problem. Unfortunately, sufficient experiments have not been made with this new system to see if this is true. Further studies on HyperMirror-II, such as extraction and analysis of problems, will be made later. Fig.17 A conversation using a map on HyperMirror-II SITUATION FOR EFFECTIVE HYPERMIRROR USE The feeling of being in the same room reduces psychological barriers between participants. Communication with HyperMirror offers a softer, "athome" feeling. There are now more situations where this system is effective, even in business discussions. In interior decorating, parts assembly in schools and plants, and construction site, the site and content are closely related. HyperMirror is effective in situations where it is difficult to separate the two. If the supervisor or customer cannot be present, they indicate their preferences looking at items being constructed. They can appropriately indicate points that are unsatisfactory and effectively communicate their detailed ideas to those doing the work. Because HyperMirror operates in real time, the contractor can make preferences known on site. Results then serve as a base when the contractor expresses ideas in more detail. It is also possible to instantaneous convey suggestions from the site to the contractor. Also, by switching to an idealized image, it is easier to exchange information required for conversation. The result is a more complete, thorough exchange with a higher level of concreteness. CONCLUSION HyperMirror is effective in communication because it is easy to understand and has symmetry. Like with the telephone, what I see is also visible to others. This spontaneously creates a communication rule that objects visible only to me are excluded from conversation. If such objects must be mentioned, behavior such as explanations and close-up will be used spontaneously. When we realize an object, we understand in relation to other objects, not alone. Self-image--that is, mirror reflection--is information most familiar to us. With a reflection displayed, we understand everything involved in conversation in the structure related to ourselves. Display of reflection also makes us aware of participation in conversations, giving a sense of responsibility to respond. This system performs image processing to create HyperMirror images. Produced images are not present in reality, but the correspondence between images and the real world is very simple and understandable to most people. By making full use of the latest computer graphics, impressive images impossible in this world will be produced. It will also be possible to convert the image of a face looking at a screen to an image in the camera gaze direction. However, these applications are not priorities for fully implementing this system. This is because the system design principle is that we fully utilize it based on a understanding of it. If it contains any obscurity, even slight, the reliability and value of video image information will be reduced to where communication cannot take place. Back to the initial subject, telephones may not transmit images, but this does not deny their vast usefulness. We would like to view any inconveniences with HyperMirror as potential advantages, in the same way. In comments by HyperMirror users, those living away from home viewed HyperMirror very favorably because it provided shared (although virtual) space. The system would best be set up as a public facility to avoid large numbers of unexpected anonymous visitors to a "home" setting. Some participants at a site with covered by thick blue paper felt that they had actually been "transferred" to real space where others were present. Others said that they would enjoy going hiking with friends via HyperMirror if they were hospitalized, for example. We plan to continue future work with HyperMirror, and welcome your comments and suggestions. REFERENCES 1. Anstis, S., Mayhew, J., Morley, T., "The perception of where a face or television 'portrait" is looking, American J. Psychology, 82, (1969) 2. Dourish, P., Adler, A., Bellotti, V. and Henderson, A., Your Place or Mine? Learning from Long-Term Use of Video Communication, Technical Report EPC (1994) 3. Gaver, W., Sellen, A., Heath, C. and Luff P., One is not enough: Maltiple views in a media space. Proc. INTERCHI'93, (1993) 4. Greenhalgh, C. and Benford, S., Massive: A collaborative virtual enviroment for teleconferencing. ACM Transactions on Computer-Human Inteaction, 2.3, September1995, (1995).

10 5. Hall, E., The Hidden Dimension, (1970 In Hidaka, T., and Sato, N., trans. in Japanese), Misuzu Shobo(1966) 6. Heath, C. and Luff, P., Disembodied Conduct: Communication through video in a multi-media environment, Proc. CHI'91, (1991) 7. Heath, C. and Luff, P., Media Space and Communicative Asymmetries: Preliminary Observations of Video-Mediated Interaction", Human- Computer Interaction, 7(3), , (1992). 8. Ishii, H., TeamWorkStation: Towards a Seamless Shared Workspace, Proc. CSCW '90, 13-26,(1990). 9. Ishii, H. Kobayashi, M. and Grudin, J., Integration of Interpersonal Space and Shared Workspace:ClearBoard Design and Experiments, ACM Transactions on information Systems(TOIS), ACM, 11-4, (1993) 10. Kendon, A, Some functions of gaze direction in social interaction. Acta psychologica, 26, 22-63(1967) 11. Krueger, M., Artificial Reality II, Addison-Wesley Pub.(1991) 12. Kuzuoka, H. and Nakada, T., Assisting Remote Instruction Using Copied Reality.Advances in Human Factors/Ergonomics, 21B, (1997) 13. Kuzuoka, H., Kosuge, T. GestureCam:A video Communication System for Sympathetic Remote Collaboration. Proc. CSCW' (1994) 14. Maes, P., Blumberg, B., Darrell, T. and Pentland, A., The alive system: Full-body interaction with animated autonomous agents. ACM Multimedia Systems, 5: (1997) 15. Mandeville, J. et al., GreenSpace: Creating a Distributed Virtual Environment for Global Applications, In Proceedings of IEEE Networked Virtual Reality Workshop(1995) 16. Morikawa, O. and Maesako, T., HyperMirror: a Video- Mediated communication system, CHI97 extended abstracts, (1997) 17. Morikawa, O., Difference of Gaze Perceptions Between on Face-to-face and on Video communication(in Japanese), Progress in Human Interface, 6, 13-20(1997) 18. Morikawa, O., Effects of displaying the reflected image of users(in Japanese), Proc. of the 14th Annual Meeting of the Japanese Cognitive Science Society, (1997) 19. Morikawa, O., Gaze Performance in Video communication, A bridged Proc. HCI International 97, 24-24(1997) 20. Morikawa. O., A human processing model for explaining the effects of media in communication (in Japanese), Information Processing Society of Japan SIG Notes, HI63-6, 41-47(1995) 21. Okada, K., Maeda, F. Ichikawa, Y., and Matsushita, Y., Mulyiparty Videoconferencing at Virtual Social Distance:MAJIC design, Proc. CSCW'94, (1994) 22. Sellen, A., Speech patterns in video-mediated conversations. Proc. CHI'92, 49-59(1992). 23. Sommerer, C. and Mignonneau, L., "MIC Exploration Space, " in Siggraph'96 Visual Proceedings, 17-17(1996). 24. Sugawara, S., et al., InterSpace: Networked Virtual World for Visual Communication, IEICE transactions on Information and Systems, E77-D(12), (1994) 25. Tang, J. and Minneman, S., VideoDraw: A Video Interface for Collaborative Drawing", ACM Transactions on information Systems(TOIS), ACM, 9-2, (1991) 26. Tang, J. and Minneman, S., VideoWhilteBoard: Video Shadow to Support Remote Collaboration", Proc. CHI'91, (1991) 27. Wellner, P., Interaction with the Paper on the DigitalDesk, Communication of the ACM, 36.7, (1993) 28. Yoshida, A., Kakuta, J., Research study on the communication by telephone in Japanese university students(in Japanese), Proc. of 8th Symposium on Human Interface, (1992)

11