URL: DOI: /ROMAN

Transcription

1 Kaiko Kuwamura, Takashi Minato, Shuichi Nishio, Hiroshi Ishiguro, "Personality Distortion in Communication through Teleoperated Robots", In IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), Paris, France, pp , September, URL: DOI: /ROMAN IEEE Copyright Notics: 2012 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

2 Personality Distortion in Communication through Teleoperated Robots* Kaiko Kuwamura 1,2 Takashi Minato 2 Shuichi Nishio 2 Hiroshi Ishiguro 1,2 Abstract Recent research has focused on such physical communication media as teleoperated robots, which provide a feeling of being with people in remote places. Recent invented media resemble cute animals or imaginary creatures that quickly attract attention. However, such appearances could distort tele-communications because they are different from human beings. This paper studies the effect on the speaker s personality that is transmitted through physical media by regarding appearances as a function that transmits the speaker s information. Although communication media s capability to transmit information reportedly influences conversations in many aspects, the effect of appearances remains unclear. To reveal the effect of appearance, we compared three appearances of communication media: stuffed-bear teleoperated robot, human-like teleoperated robot, and video chat. Our results show that communication media whose appearance greatly differs from that of the speaker distorts the personality perceived by interlocutors. This paper suggests that the design of the appearance of physical communication media needs to be carefully selected. I. INTRODUCTION People usually want physical information, which includes such non-verbal information as gaze behavior and bodily gestures, of their tele-communication counterparts. This is because non-verbal information plays a crucial role in communication [1], [2]. Recently, teleoperated robots are being created as communication media with physical existences that can interact physically to transmit a speaker s existence to remote places [3]. Compared with telephones or video chat, communication media with physical existences provide physical information and a feeling of being with the speaker who operates the medium from a remote place. Many physical communication media have been invented that resemble cute animals or imaginary creatures [4], [5], [6]. Such appearances are generally determined to make them attractive at first glance. However, if a physical medium has an appearance different from a human being, people may feel like they are talking to an imaginary character rather than a person in a remote place who is talking through it. It is obvious from the studies of Goetz et al. [7] and Komatsu et al. [8] that the appearance of a robot makes people imagine its abilities and performance. Such imagination can also happen to physical communication media. That is, the information transmitted through communication media can be influenced by their appearance. The transmission characteristics depend on such features of the physical media *This research was supported by JST, CREST. 1 Graduate School of Eng. Science, Osaka Univ., Machikaneyama-cho 1-3, Toyonaka-shi, Osaka, Japan 2 Hiroshi Ishiguro Laboratory, Advanced Telecommunications Research Institute International(ATR) Hikaridai, Keihanna Science City, Kyoto , Japan Fig. 1. Distortion by appearance of communication medium as ability imagined from their appearance. For example, if a physical medium resembles a baby, interlocutors may partially imagine the speaker to behave as a baby. If the appearance of the physical media is not appropriate for the communication situation (e.g., the conversation content and the speaker in a remote place), it might fail to properly transmit the speaker s information and to detract from a natural conversation. For example, a conversation sharing the feeling of being together might be missing. Rutter et al. [9] experimented on mediated comparisons by focusing on visual communication and physical presence. They compared face-to-face, audio only, a curtain condition in which participants sat in the same room but without visual communication, and video chat communication to estimate the effect of the differences in the amount of information transmitted by the communication media in the conversation. Their result showed that the more cueless the setting (fewer visual communication and physical presence cues), the more task-oriented, depersonalized, and unspontaneous the conversations became. In this paper, we focus on the issue in tele-communication due to the appearance of physical communication medium. Although Rutter et al. mentioned that the feeling of talking to a counterpart decreases in some media, they did not mention issues owing to the appearance of physical communication media. We believe that differences of appearance between speakers and physical media can cause confusion because interlocutors cannot identify with the communication media as the operator in the remote place. In other words, the personality of the media per se (personality of the media imagined from the appearance) is mixed with the remote speaker s personality that is transmitted through the media (Fig. 1). Many studies have addressed the effects of avatar appearances in virtual worlds. For example, Merola and Pena revealed that an avatar s appearance can make its user more

3 negative, confident, aggressive, or friendly[10]. People using attractive avatars are more likely to behave intimately with others and make different decisions depending on whether their avatars are taller or shorter than average [11]. People rely on information provided by an avatar s appearance and decode it by connecting cues to broader social knowledge [12]. These studies focused on the effect of an avatar s appearance on communication in virtual worlds. These same effects are likely to occur in tele-communications through physical communication media in the actual world. In this paper, the following is our hypothesis: Physical communication media whose appearances differ from human beings distort the operator s personality more than physical communication media whose appearances resemble human beings. We compare the following communication media with different appearances to verify our hypothesis: A physical communication medium that looks like a stuffed-bear similar to RobotPhone [4]. Telenoid [13] (Fig. 6(b)), which is a teleoperated android that resembles a neutral person that can transmit any operator s presence. A video chat that transmits an image of the operator s appearance. Through the experiments, we revealed that the appearance of the physical medium differing from human beings causes distortion in tele-communication, suggesting the utility of an appearance similar to human beings. This information should be applied to the proper design methodology of physical communication media. II. METHODS In our hypothesis, physical communication media with different appearances from humans partially fail to transmit the speaker s original personality. It seems adequate to test our hypothesis by comparing speaker s impression in face-toface conversation and in the mediated conversation. However, it is difficult to compare personality judgments for the same operator within/between participants. It is difficult to talk with the same person several times while maintaining the same mental state, and also, judging personality is strongly subjective. To avoid these problems, we evaluated the failure to transmit personality without directly comparing the judged personalities in different conversations. If depersonalization happens, the participant feels that he is sometimes talking with the speaker and sometimes talking with the physical medium itself, as described in Fig. 1. After the conversation, the participant cannot clearly judge the speaker s personality because the speaker s impression will be mixed with the impression of the physical medium. We, therefore, measured the inconsistencies in questionnaire results that ask about the speaker s personality. Another factor that we have focused on is conversation situation. We presumed that the degree of distortion in tele-communication depends on which situation the conversation is held. To learn which situations cause distortion, we set up several types of situations and interactions. From this experiment, we learned which communication media are appropriate for different situations. A. Questionnaires In this experiment, we used the Japanese Big Five personality test, a 60-item questionnaire that represents five parameters of personality (extraversion, neuroticism, openness to experience, agreeableness, and conscientiousness) that was developed by Wada and translated from the Adjective Check List by Gough and Heilbrun [14], [15]. Each parameter has twelve different items (adjective checks) concerning the parameter. All items are rated on a 7-point scale ranging from 1 (disagree strongly) to 7 (agree strongly). Although this questionnaire had been designed to test one s own personality, it can also be used for testing personalities of others, as Lauraetal. [16] did for evaluating the person in photographs. B. Evaluation If the speaker s personality transmitted through communication medium is distorted, the answers to the twelve items in the Big Five test will lack consistency. To evaluate the consistency of the questionnaire answers about speaker s personality, we used Cronbach s alpha, a coefficient of reliability [17]. Cronbach s alpha estimates the reliability of questionnaires by checking the consistency of answers. If the alpha coefficient of the participant s answers to the Big Five 1 questionnaires about speaker s personality is low, we can infer that the participant failed to correctly estimate the speaker s personality. That is, the medium cannot correctly transmit personality. In this way, we can measure the effect of physical media on the transmitted personality and avoid the problem of controlling the mental state and subjectivity on the answers. George and Mallery [18] provided the following criteria of Cronbach s alpha: α 9 excellent, 9 > α 8 good, 8 > α 7 acceptable, 7 > α 6 questionable, 6 > α 5 poor, and 5 > α inacceptable (α: Cronbach s alpha). With Cronbach s alpha, we can evaluate whether the consistency of the answers is sufficiently high without direct comparison among the judged personalities on different conversations. Cronbach s alpha is, therefore, suitable for our experiment. The benefit of cute-appearance media is that they themselves provide attractiveness and a relaxed conversation for users. To compare the relaxed state we measured nervousness by asking whether the participant felt stress on a 7-point scale after each conversation. C. Communication Media We used three kinds of communication media: a physical communication medium that resembles a stuffed-bear, which is an appearance far from the operator s appearance, similar to RobotPhone [4]; a physical communication medium with a human-like appearance; and a video chat system that lacks physical existence but transmits an exact image of the operator s appearance (Fig. 2). 1 The Japanese Big Five test had been validated to have high consistency (α > 0.84) in tests with 350 university students [14].

4 (a) Stuffed-bear robot Fig. 2. (b) Telenoid (c) Video Communication media used in experiments Fig. 4. Fig. 3. Teleoperation system for Telenoid and the stuffed-bear robot We used Telenoid [13] as a human-like physical communication medium. It is a teleoperated robot designed to represent a minimalistic human; at first glance, it is easily recognized as a human without specifically resembling a particular person. To control the size and motion of the physical media, we put a stuffed-bear costume on a Telenoid so that their sizes and shapes were about the same. We used a 27-inch monitor in the portrait orientation for the video chat so that the monitor size was close to the robots. Telenoid s teleoperation system is shown in Fig. 3. Telenoid s head motion is synchronized with the operator s head motion. The head motion captured by a web camera is recognized by image processing and converted to motor commands for Telenoid motion. Telenoid also moves its mouth and talks like the operator. Speech-driven lip motion generation, which generates lip motion from the vocal information of the operator, was used to control Telenoid s jaw [19]. The stuffed-bear robot was teleoperated by the same system. D. Experimental Room Figure 4 shows the equipment alignment in the experimental room. The operator can see the entire view of the room through the video camera to recognize both the operating robot s and the participant s movements (Fig. 5). The operator s voice, generated from the communication medium, should have the same quality and volume among all three media. Therefore, we used the same speakers placed on the side of communication media. The operator can hear the sound in the experimental room by a microphone placed under the participant s chair. Experimental setup Fig. 5. A screen capture of the operator interface (left: lip motion generation system, center: view of the experimental room, right: head motion recognition system) E. Procedure To examine how dependable the distortion is in the conversation situation, we arranged three types of conversation situations each with different experimenters about different topics through the same medium: relaxed conversation in which both the participant and experimenter talked; selfintroductions in which only the experimenter talked; and interviews in which the participant did most of the talking (Table I). For the first conversation, we arranged 10 minutes of free talk with no restrictions to allow the participants to get used to the communication medium. In the second conversation, the experimenter introduced himself for two minutes (since the participant passively listens in this situation, we arranged shorter interactions than in the first conversation). For the third conversation, the experimenter interviewed the participant about the experiment. In the interview, the experimenter wore a business suit to add to the situation s verisimilitude and to slightly pressure the participant in contrast with the second conversation. The third conversation averaged 151 seconds. Finally, we conducted brief interviews with the participants and asked for their impressions of the communication medium and the experimenter through the medium. In the second conversation, we use a pre-recorded movie for video chat and pre-recorded motions and sounds for the physical communication media to control the conversation among the participants, since there is no interaction between the experimenter and the participant. In the third conversation with video chat, the experimenter asked prepared questions

5 TABLE I CONVERSATION SITUATION First situation Second situation Third situation Free Talk Self-Introduction Interview university student graduate student responsible person 10 minutes 2 minutes 151 seconds for every participant. To control the conversations with the physical communication media, we used the operator s motion and sound recorded in the video chat situation for interviews. The process of the experiment is as follows: 1) Participant receives an explanation, signs the consent form in the waiting room, and moves to the experimental room. 2) The first conversation begins: 10-minute free talk with an university student. 3) After ten minutes, the experimenter (a different person from the operator) stops the conversation, hides the medium behind a curtain, and asks the participant to answer the questionnaire about the Big Five and nervousness. 4) After the participant finishes, the curtain is removed, and the second conversation begins: 2-minute selfintroduction by a graduate student. 5) After the second conversation, the experimenter stops the conversation, hides the medium again, and asks the participant to answer the questionnaire. 6) After the participant finishes, the experimenter explains that the person responsible for the experiment wants to interview the participant about the experiment and that he will talk through the communication medium since he is in the operator s room. Then the third conversation begins: interview. 7) After the third conversation, the experimenter stops the conversation, hides the medium, and asks the participant to answer the questionnaire. 8) After the participant finishes, the experimenter asks some questions. We asked about impressions of the communication medium and whether the participant noticed that the second or third conversation used prerecorded movies and motions. A. Target Participants III. RESULTS Participants were 46 university students (34 males and 12 females)whoseaverageagewas21.0years(sd = 2.0).Five who answered incorrectly were omitted from analysis. We collected the following datasets: 15 for the stuffed-bear robot, 15 for Telenoid, and 11 for the video chat. All participants provided written informed consent in accordance with the ethics approval 2. 2 This experiment was approved by the ethical committee of Advanced Telecommunications Research Institute International (No ) B. Experimental Results EachparameteroftheBigFivetestusedinourexperiment had high consistency (α > 0.80)(Table II). However, the stuffed-bear medium had low consistency (α < 0.70) on some parameters under some situation(table III). According to George and Mallery, a coefficient under 0.7 means the questionnaire has less reliability [18]. We infer that the personality transmitted through the physical media with the stuffed-bear was distorted under certain situation. Figure 6 shows the result of nervousness for each medium 3. There were no significant differences among the communication media. Two-way (media situation) ANOVA revealed only significant effect of situation (F(2,64) = 9.43, p < 0.001). Friedman test for each media showedsignificanceinstuffed-bearrobot(χ 2 (2) = 7.88,p < 0.05) and Telenoid (χ 2 (2) = 9.68, p < 0.01), and multiple comparisons by the Scheffe s paired comparison method showed significant difference between self-introduction and interview for stuffed-bear robot (χ 2 (2) = 8.68, p < 0.05), between free talk and Interview (χ 2 (2) = 9.12, p < 0.05) and between self-introduction and interview (χ 2 (2) = 9.78, p < 0.01) for Telenoid 4. IV. DISCUSSION In this experiment, we found that participants impression for the stuffed-bear physical communication medium had low consistency (α < 0.7) even though the Big Five questionnaire proved to have high consistency. When we asked the participants about their impressions of the stuffedbear robot, some described it as cute and others said they did not feel as if they were talking to a person. If such feeling is strong and all of the items in the questionnaire are uniformly biased by this feeling, the resulting consistency in answers will not decrease. However, some parameters had less consistency. We infer that low consistency in the result means that the participant had mixed feelings about talking to the operator who was influenced by the communication medium. As the result, the stuffed-bear robot in the selfintroduction and interview situations seems to invoke a low consistency below 0.7. The stuffed-bear s appearance might had reminded participants of the famous friendly cartoon bear that has many friends. Extraversion and Agreeableness were probably distorted by such imagination. We can also infer that such distortion effect strongly occurs in situations with few interactions (self-introductions and interviews) because participants do not obtain much information about the speaker from the conversation. If the self-introduction and interview situations were as long as the free talk, the consistency might increase. However, even short interactions produced enough consistency in some parameters. The result with the stuffed-bear robot is not just reflected in the amount of interaction. 3 Six who noticed that the second or third conversation used pre-recorded movies and motions were omitted from analysis. 4 We used non-parametric tests since normality within each media were rejected by the Shapiro-Wilk test.

6 TABLE II RESULT OF COEFFICIENT ALPHA All Free Talk Self-Introduction Interview TABLE III COEFFICIENT ALPHA ON EACH SITUATION (*: α < 0.7) (a) Stuffed-bear robot Free Talk Self-Introduction * Interview 0.663* (b) Telenoid Free Talk Self-Introduction Interview (c) Video chat FreeTalk Self-Introduction Interview (a) Stuffed-bear robot (b) Telenoid (c) Video chat Fig. 6. Multiple comparisons of nervousness (*:p < 0.05, **:p < 0.01) In this experiment, the physical communication media with human-like appearances less distorted the personality that was transmitted from the operator at remote. However, not all human-like media have identical results, possibly owing to the appearance of Telenoid that represents neutral humans. There are other human-like robots, including Geminoid [20] or mechanical humanoid robots in science fiction. Geminoids are android robots that are made to completely resemble their human source. Therefore, if the operator s appearance is different from the Geminoid s appearance, distortion may occur, as in the stuffed-bear robot. In fact, when we talk with a person through Geminoid HI-1, we cannot recognize whether the speaker is an acquaintance [21]. Also, in our experiment results, the video chat seems to transmit personality very well as shown in the high coefficient values. Although the video chat transmits much information, rich information has negative effects in some

7 situations. Figure 6 shows that although video chat tends to produce consistently high feelings of anxiety, the two physical communication media tend to produce different feelings of anxiety owing to the situations. If a communication medium transmits too much information, the interlocutor might experience stress from the operator. When we want to exert pressure in a conversation, we should use video chat, and we should use physical communication media when we want to avoid causing pressure. To disguise our personality or make the interlocutor imagine a friendly person, using a stuffed-bear robot is better than either video chat or Telenoid. Suitable situations exist for each communication medium. In this paper, we did not focus on the effect to operators. In Yee and Bailenson s study [11], users of tall avatars were more confident in a bargaining task while those assigned short avatars were more likely to accept unfavorable decisions. Operators using physical communication media can be influenced by the appearance of the physical media [22]. If the appearance of the physical medium is a cute stuffed-bear, the operator might talk nicely or sweetly. In this experiment, we used pre-recorded movies and motions as much as possible to remove any effect on the operator. We did not compare personalities measured in face-toface conversations. We must compare judged personalities in mediated, face-to-face situations to determine how much the mediated conversation is influenced compared with a faceto-face conversation. Furthermore, we must include another communication media where a human avatar, a media having an appearance of human with no physical presence, be present in a virtual environment. Future work will investigate these issues to suggest a design methodology for physical communication media. V. CONCLUSION We compared three communication media to measure the effect on the speaker s personality that was transmitted through the physical communication media by regarding their appearance as a function to transmit the speaker s information. When the communication media have different appearances from the operator, the personality of the operator transmitted through the media is distorted and the interlocutor get confused. We speculate that this is because the personality that comes from the physical media per se distorts the original personality. Such problems can be solved by giving the communication media a human-like appearance. Video chat also did not distorted the operator s personality. Thus, video chat tends to produce consistently high feeling of anxiety if compare with the physical communication media. When we want to avoid causing pressure, we should use physical communication media. We focused on the appearance of communication media. We discussed the problem of the appearance of communication media and found the usefulness of having humanlike appearances. We believe that this study will help design communication media appropriate for various situations. REFERENCES [1] A. Mehrabian, Silent messages. Belmont, California: Wadsworth Pub., [2] B. A. Kendon, Do gestures communicate? a review, Research in Language and Social Interaction, vol. 27, no. 3, pp , [3] S. Tachi, Telexistence: Next-Generation Networked Robotics, in Telecommunication, Teleimmersion and Telexistence II(S. Tachi ed.). IOS Press, [4] D. Sekiguchi, M. Inami, and S. Tachi, RobotPHONE: RUI for Interpersonal Communication, CHI2001 Extended Abstracts, pp , [5] K. Goris, J. Saldien, and D. Lefeber, The huggable robot probo, a multi-disciplinary research platform (long), Eurobot 2008 Conference, [6] W. D. Stiehl, J. K. Lee, C. Breazeal, M. Nalin, A. Morandi, and A. Sanna, The huggable: a platform for research in robotic companions for pediatric care, IDC 09 Proceedings of the 8th International Conference on Interaction Design and Children, pp , [7] J. Goetz, S. Kiesler, and A. Powers, Matching robot appearance and behavior to tasks to improve human-robot cooperation, in Proceedings of the IEEE Workshop on Robot and Human Interactive Communication, [8] T. Komatsu and S. Yamada, Adaptation gap hypothesis: How differences between users expected and perceived agent functions affect their subjective impression, Journal of Systemics, Cybernetics and Informatics, vol. 9, no. 1, pp , [9] D. R. Rutter, G. M. Stephenson, and M. E. Dewey, Visual communication and the content and style of conversation, British Journal of Social Psychology, vol. 20, no. 1, pp , [10] N. Merola and J. Pena, The effects of avatar appearance in virtual worlds, Journal of Virtual Worlds Research, vol. 2, no. 5, [11] N. Yee and J. Bailenson, The proteus effect: The effect of transformed self-representation on behavior, Human Communication Research, vol. 33, pp , [12] K. Isbister, Better Game Characters by Design: A Psychological Approach(The Morgan Kaufmann Series in Interactive 3D Technology), 1st ed. Morgan Kaufmann, [13] K. Ogawa, S. Nishio, K. Koda, G. Balistreri, T. Watanabe, and H. Ishiguro, Exploring the natural reaction of young and aged person with telenoid in a real world, Journal of Advanced Computational Intelligence and Intelligent Informatics, vol. 15, no. 5, pp , [14] S. Wada, Construction of the Big Five Scales of personality trait terms and concurrent validity with NPI, Japanese Journal of Psychology, vol. 67, no. 1, pp , 1996, [in Japanese]. [15] H. Gough and A. Heilbrun, The adjective check list manual. Consulting Psychologists Press, [16] L. P. Naumann, S. Vazire, P. J. Rentfrow, and S. D. Gosling, Personality judgments based on physical appearance, Personality and Social Psycholoty Bulletin, vol. 35, no. 12, pp , [17] L. J. Cronbach, Coefficient alpha and the internal structure of tests, Psychometrika, vol. 16, no. 3, pp , [18] D. George and P. Mallery,SPSSforWindowsStepbyStep:ASimple Guide and Reference, 11.0 Update, 4th ed. Allyn and Bacon, [19] C. T. Ishi, L. Chaoran, H. Ishiguro, and N. Hagita, Speech-driven lip motion generation for tele-operated humanoid robots, International Conference on Auditory-Visual Speech Processing (AVSP2011), pp , [20] S. Nishio, H. Ishiguro, and N. Hagita, Geminoid: Teleoperated android of an existing person, in Humanoid Robots: New Developments, A. C. de Pina Filho, Ed. Vienna, Austria: I-Tech Education and Publishing, Jun 2007, pp [21] H. Sumioka, S. Nishio, E. Okamoto, and H. Ishiguro, Doppel teleoperation system: Isolation of physical traits and inteligence for personality study, In Annual meeting of the Cognitive Science Society (CogSci2012), August 2012 (to appear). [22] I. Straub, S. Nishio, and H. Ishiguro, Incorporated identity in interaction with a teleoperated android robot: A case study, in IEEE International Symposium on Robot and Human Interactive Communication(RO-MAN), Viareggio, Italy, Sep 2010, pp