The Relationship between the Arrangement of Participants and the Comfortableness of Conversation in HyperMirror

Transcription

1 The Relationship between the Arrangement of Participants and the Comfortableness of Conversation in HyperMirror Osamu Morikawa 1 and Takanori Maesako 2 1 Research Institute for Human Science and Biomedical Engineering, AIST, Higashi, Tsukuba, Ibaraki, , Japan 2 Faculty of Human Sciences, Osaka University, 1-2 Yamadaoka, Suita, Osaka , Japan morikawa.osamu@aist.go.jp, maesako@mcgyver.hus.osaka-u.ac.jp Abstract. HyperMirror is a new type of video conversation system which does not simulate face-to-face conversation in real space. In real space, people may feel that a relative positional relationship to the other person is comfortable and sometimes that it is not. They seem to feel a similar relationship also in HyperMirror. In this paper, we observe the relationship between arrangement of participants on the HyperMirror screen and comfortableness of conversation by changing position of the camera and the participants' standing positions. We find two facts; in the HyperMirror screen, they feel at ease to speak when they are near or look toward their partner, and it is more important that they look toward their partner than that they are looked toward. Introduction The HyperMirror displaying one's self-image is a new type of video communication system, it does not imitate face-to-face conversation [8, 9]. In a HyperMirror conversation (Fig.1) all participants see the same image, displaying themselves and their partners in the same room on the screen. So that, the positional arrangement of the participants and the items displayed on the screen can be used in the communication. For example, a pointing at objects on the screen is possible. Moreover, there are no walls between the participants, they may move freely within the conversation space on the HyperMirror screen. Through the experience with the HyperMirror for four years, it appears that there are arrangements in which the participants feel at ease to speak, while in some arrangements they find it uncomfortable to speak. In the participants' report, they said, "As you don't look at me, I feel you don't talk to me although you seem to talk to me." "I don't feel you speak heartily." It seems to be a similar sense to a disagreement of gaze. In this paper, we study the relationship between the arrangement of participants on the HyperMirror screen and the comfortableness of conversation by changing position of camera and partner's standing position.

2 The relationship between the standing point and the facing direction on the HyperMirror screen Usually in video conversation, people look at their partner on a video monitor when they speak. Therefore, when the listener on the screen is near the speaker's camera, the full face of the speaker will be shown. But if the listener is displayed, the farther from the speaker's camera, the more of the speaker's profile will be shown on the screen. In HyperMirror, the self s mirror reflection is presented with his/her partner on his/her own screen. Let's assume that there is a camera on the left side of a screen, on which there are two listeners' (A and B s) reflections (Fig.2). When a speaker looks at A's reflection, the speaker's reflection is profiled to the right a little. On the HyperMirror screen, when the speaker's mirror reflection is on the left of A's reflection (inside of zone-1), it looks to see listener A or B. The difference between looking toward A and B shows the difference of the speaker's face direction. When the speaker looks to A's reflection, the speaker's reflection is almost full face. When the speaker looks to B's reflection, the speaker's reflection is profiled a little. When the speaker's reflection is in the zone-2 (between A's reflection and B's), the speaker's reflection always looks to see B even though the speaker looks at A or B. Even though the speaker looks at A's reflection, the speaker's reflection looks away from A. When the speaker's reflection is in the zone-3 (on the right side of A's and B's reflections), the speaker's reflection looks away from A and B. A B zone-1 zone-1 zone-2 zone-3 zone-2 zone-3 Fig. 1. HyperMirror Fig. 2. Presentation zone and its meaning Experiment 1 In HyperMirror, since one's own image is displayed on a screen, the facing direction has different meanings depending on where the self-image appears on the screen. We carried out experiments examining how the comfortableness of talking varies depending on the relative positions of the listener and the position of the displayed self-image. The subjects were 36 (18 female) people, their ages ranging between 10 and 60 years. Each experiment was conducted with two acquainted subjects. The equipment was a 2 site chromakey version of the HyperMirror (Fig.3). A projector, a screen and a camera were installed in both rooms, the background of the

3 second room was a simple blue curtain for chromakey synthesis. The screens were 90cm high, 120cm wide, standing 90cm above the floor. The cameras were placed 150cm high in the two rooms, on the left side of the screen in Room-1, and on the right side of the screen in Room-2. The distance between the camera's location and the center of the screen was 70cm. The standing positions of the subjects were 300cm in Room-1 and 360cm in Room-2 from the center of the screen. Three positions were examined in each room, one facing the center of the screen, one 80cm to the left and one 80cm to the right from the center. In this paper we denote the left, center, and right positions by l, c, r (in Room-1), and L, C, R (in Room-2), respectively L' C' R' Room- 1 chromake y mixer Reverser Room l c r L C R L C R Fig. 3.. HyperMirror system used experiment. Fig. 4. Angle of the faces of the reflection Although the distances between the camera and the subjects were different between the two rooms, we adjusted the camera to project images of same size and at the same position on the two screens. The cameras were adjusted to show images of 210 cm width with the 3 standing positions at 50 cm intervals on both screens. The precision of both cameras and screens were of NTSC-TV quality. The HyperMirror video signal was a chromakey synthesis of the video signals from the cameras in each room. It was sent to each room and the reverse image of it was projected on each screen. As the distances to camera are different between in Room-1 and Room-2 the angles to full face are different between them. Let us assume that the subject at R (in Room-2) sees the reflection at L' on the screen who is standing at l (in Room- 1)(Fig.4). The camera in Room-2 is 360cm forward and 10cm = (80-70) left. On this occasion when subject at R looks at the camera, the reflection is a full face, so that the face of reflection turns left a little. By simple calculation, the angle is 18 degrees (full face is 0 degrees). So, the mirror reflection of the subject at R looks to left side of the screen in 18 degrees. Similarly, the angles of the faces of the reflection at R, C or L when s/he sees L', C' or R' can be calculated. By calculation, they are 3, 13 and 22 degrees in Room-1 and 3, 11, 18 degrees in Room-2. To make the subjects familiar with the environment, after a brief explanation of the HyperMirror system, they experienced the handshaking over the screen, finger pointing at objects in the shared space of the HyperMirror environment, and were allowed to freely communicate with their partners for 5 minutes. Next, the subjects were instructed to move into the directed positions, shake hands, greet each other over the screen and rate the comfortableness of conversation in the given arrangement on a

4 7-degree-scale (very natural / comfortable, natural / comfortable, a little natural / comfortable, neither natural / comfortable nor unnatural / uncomfortable, a little unnatural / uncomfortable, unnatural / uncomfortable, very unnatural / uncomfortable). The experiments were carried out in six different arrangements of standing positions, in random order. After the evaluation of six positions, the subjects expressed their impressions freely. Results After translating the subjective evaluation to a scale running from -3 to +3 (-3 = unnatural or uncomfortable, +3 = natural or comfortable), statistical analyses were performed. Analysis showed that subjects gave a positive evaluation when the image of a subject in Room-1 was on the left side of the partner's image on the screen {lr,cr,lc}. In these cases, they were facing their partner on the screen. Negative evaluations were given when a image of a subject in Room-1 was on the right side of the partner's image on the screen {rl,rc,cl}. In these cases, they did not face their partner on the screen. The worst evaluations were given for the cases where they were standing far away from their partner. The evaluations can be divided into 3 categories (see Figure 5): uncomfortable for speaking {rl}, mixed feeling {rc,cl}, and comfortable for speaking {lr,cr,lc}. The differences among the three groups were statistically significant (Student's t-test, p less than or equal to 1%). There was no statistical evidence for differences within each group total Room- 1 Room- 2 rl rc cl lr cr lc c R Fig. 5. Subjective evaluations of experiment-1 Comparing Room-1 and Room-2, subjective evaluations tended to be more extreme for Room-1 than for Room-2. In Room-1, the distance from the subject to the camera was shorter than in Room-2, causing a greater change in facial angle between the three standing positions than for Room-2. This was reflected in the subjective evaluations. Room-1 tended to get a lower evaluation on the uncomfortable condition {rl}, where subjects are separated in left to right on the screen and see outside each other, than Room-2, and higher evaluations on the comfortable conditions {lr,cr,lc} than Room-2.

5 Experiment 2 According to previous experiment, there were higher naturalness or comfort scores when subjects look to their partner than when they look to away from their partner. Then we experiment on the comfortableness when the directions of both people are same. This experiment was carried out in the same rooms as the previous one, only the camera in Room-2 was placed on the left side of the screen this time. This way, all participants were displayed facing to the right regardless of their standing point. Therefore, it was impossible for two participants to be displayed facing each other. In all cases, the participant on the left side was shown facing the participant on the right, who again was displayed turning slightly to the right. The procedure of the experiment was the same as in the previous one. Results Translating the subjective evaluation to a 7-degree-scale for statistical analysis was done in the same way as for the previous experiment (Fig.6) total Room- 1 Room- 2 rl rc cl lr cr lc c R Fig. 6. Subjective evaluations of experiment-2 The tendency of the subjective evaluations was different from the previous experiment, as can be seen in Fig.6. The evaluations did not seem to depend on whether the subjects were on the left or right side of the screen, but rather, on the distance of the subjects from each other. When the separation distance on the screen was large {rl,lr}, the subjective evaluations were low. When subjects were standing close together, the evaluations were lower when they were further from the camera {rc,cr} than when they were nearer to the camera {cl, lc}. That is, a higher score was given to a nearly full face view than to a profile. Although some of the standing position arrangements received high subjective evaluations in Experiment 2, they were lower than the natural/comfortable arrangements in Experiment 1 {lr,cr,lc}. There were no statistically significant differences between Experiment 1 and Experiment 2 because of the large variance in Experiment 2.

6 Discussion In the first experiment, the conversation was felt natural when both participants were mutually facing towards each other, no matter what the separating distance was. The conversation was felt equally natural in cases of standing side by side (80cm distance, if converted to real space) and cases of standing distantly (160cm distance, if converted to real space). However, when the participants' images were shown facing away from each other, although it was not very unnatural when the participants stood side by side, it was felt very unnatural when they stood distantly. In the second experiment, the participant standing on the left was always displayed on the screen facing the other participant, who was displayed facing out to the right (both of them were displayed facing to the right). In each arrangement, the subject standing on the left felt less unnatural than the other participant standing on the right. Moreover, the results for same role arrangements (relative left-right positions of the participants) revealed similar tendencies for both participants. In HyperMirror conversation a participant plays the roles of a speaker and an audience at a same time because his/her reflection is displayed on the screen and s/he sees it. Subjects evaluate the HyperMirror picture as audiences. In short, the evaluation of the HyperMirror picture depended greatly on how one's own image was displayed. It appears to be more important for the comfortableness of the conversation that one's own image faces the partner than the partner faces one's direction. A participant tolerates that partner's reflection does not face her/his own reflection. S/he wants that her/his own reflection faces the partner's reflection [2, 3, 7]. But it is not satisfying in some conditions. In the condition, where the participant's own reflection does not face to partner when s/he looks at the partner's reflection on the screen and s/he cannot see the screen when her/his reflection faces to partner's reflection on the screen. That is, s/he cannot play the role s/he wants. This gap between the role a participant can play and the role s/he wants to play causes uncomfortableness [1]. A gaze direction expressed by the body posture which is observed in many HyperMirror conversations seems to be based on this gap. In both experiments, the differences of evaluation scores were distinguished in Room-1, probably because the shorter distance between the participants and the screen leads to drastic changes in the angle of one's facing direction and the direction of the camera's focus. This also proved that the way one's own image is displayed has more effect on the comfortableness of conversation than the way the partner is displayed. These results reflect human behavior in sociopetal/sociofugal settings in real space. In real space, when people want to speak, they face toward the partner and choose a position to see her/him in central vision. Consequently, it is a sociopetal setting. On the other hand, when people happen to be standing together by mere chance and they don't have a desire to talk, they behave to discourage interaction and choose a position not to see the other in central vision. Consequently, it is a sociofugal setting. People choose an arrangement which best suits their desire to talk. Conversely, sometimes the arrangement makes an influence on the conversation of people. In sociofugal settings, people see their partner peripherally, whether s/he is out of their sight or not. Either way, it is easy to ignore the presence of other people, and it is natural not to have a conversation.

7 In HyperMirror communication, even if people are in a sociofugal arrangement on the screen, they can see their partner in their central vision. The visual information is equal to that of the sociopetal arrangement. That is, the following real world causal relationship that we can get through the arrangement does not hold in HyperMirror space: Sociopetal arrangement --> people see each other in their central vision --> people want to talk. Sociofugal arrangement --> people do not easily see each other in their central vision --> people avoid talking. The difficulty that people feel in talking in the sociofugal arrangement on the HyperMirror screen is not caused by physical elements of the visual information. It is caused by people's knowledge of sociofugal arrangements in the real world. The recognition of the sociofugal arrangement is a cognitive schema. People feel that they cannot talk easily in that arrangement even though they can see their partners in their central vision. Although a speaker does not always see a listener during conversation, s/he sees the listener at some important points to know the listener s statement. Similarly, a listener does not always see the speaker during conversation, s/he looks at the speaker or s/he nods his/her head to show her/his listening to or being interested in the speech. These actions are useful not only for a speaker and a listener but also for onlookers. These actions identify who is a listener and a speaker [4, 5]. This role of the actions is important when more than two people have a conversation. It may be difficult for them to identify a speaker and a listener at the moment they see the picture. But observation for a while shows who joins in the conversation. When more than two people have a conversation, a participant becomes a speaker, a listener, and an onlooker or an audience. Moreover, in HyperMirror a speaker has an audience viewpoint because s/he sees her/his own reflection on the screen during conversation, and that the speaker has such a viewpoint is shown to play an important role in smooth conversation in HyperMirror. Another example is that people try to keep an appropriate space with the partner on the HyperMirror screen [9]. This action is similar to taking personal space in the real world that causes discomfort if invaded [6]. Fig. 7. Sociopetal (upper) and sociofugal (lower) arrangements CONCLUSIONS In this paper, the relation between arrangement of participants and comfortableness of conversation is described.

8 HyperMirror is a new type of video mediated communication system different from face to face conversation in the real world. In the real world, the arrangement of participants influences the participants' sight. Then in sociopetal arrangement the conversation may take a lively turn, while in sociofugal arrangement the conversation is controlled automatically with the decrease of common space of their sight. But when the arrangement of participants doesn't influence the participants' sight in HyperMirror space, an arrangement has a similar meaning to participants as it has in the real world. In Experiment 1, people felt comfortable talking when they faced their partner, whether they were close together or far away from each other. When they were not looking at their partner, they must be near to each other on the screen to feel comfortable. This feeling is similar to the sociopetal arrangement in the real world. In contrast, when they did not face their partner, people felt uncomfortable because of a real world sociofugal arrangement. In addition, the result of Experimental 2 showed that although it is important for participants to have their partner looking at them on the HyperMirror screen, it is more important that the self reflection looks one s partner. It turns out that the negative evaluation of the sociofugal arrangements on the HyperMirror display were not caused by the physical elements of the conveyed visual information. It is rather the recognition of the sociofugal arrangement as a cognitive scheme, which lead to uneasiness in the conversations. The HyperMirror system was designed on the principle that optimum utilization is based on an understanding of its differences from the real world. On the other hand, people tend to use their knowledge of the real world in HyperMirror space. Thus, the improving HyperMirror requires not only research of the system but also of human cognition. References [1] Argyle, M. & Cook, M. Gaze and Mutual Gaze, Cambridge University Press (1976). [2] Exline,R.V. "Explorations in the process of person perception: Visual interaction in relation to competition, sex, and need for affiliation", Journal of Personality, 3 1,1-20 (1963). [3] Exline, R.V., Gray, D., & Schuette, D. "Visual behavior in a dyad as affected by interview content and sex of respondent", Journal of Personality & Social Psychology, 1, (1965). [4] Gibson. J.J. & Pick A.D. "Perception of another person's looking behavior", American J. Psychology, 76, (1963) [5] Goodwin, C. Conversational Organization: Interaction between Speakers and Hearers, Academic Press, New York (1981) [6] Hall, E., The Hidden Dimension, (1970 In Hidaka, T., & Sato, N., trans. in Japanese), Misuzu Shobo (1966). [7] Kendon, A. Some functions of gaze direction in social interaction, Acta Psychologica, 26, (1967). [8] Morikawa, O. & Maesako, T.; HyperMirror: a Video-Mediated communication system, CHI'97 extended abstracts, (1997) [9] Morikawa, O. & Maesako, T.; HyperMirror: Toward Pleasant-to-use Video Mediated Communication System, CSCW'98, (1998).