PopObject: A Robotic Screen for Embodying Video-Mediated Object Presentations

Transcription

1 PopObject: A Robotic Screen for Embodying Video-Mediated Object Presentations Kana Kushida (&) and Hideyuki Nakanishi Department of Adaptive Machine Systems, Osaka University, 2-1 Yamadaoka, Suita, Osaka , Japan {kana.kushida,nakanishi}@ams.eng.osaka-u.ac.jp Abstract. Some studies have been conducted on 2.5D display surfaces, which displays a two-dimensional video with a curved or deformable display surface. In this study, we developed a telepresence system, which protrudes a specific part of a remote video by a 2.5D display surface. The system has a stretch projection screen and a push-out mechanism. The screen is pushed out from behind and expresses protrusion. This protrusion is aiming to express the depth information of the remote video. We expected that it enhances the remote conversation partner s presence. We supposed a conversation such as showing and explaining an object in videoconferencing, and conducted experiments in order to confirm an effect of the developed system as a telepresence system. The protrusion was in synchronization with the movement of the objects on the projected video. The results of the experiment suggested that the protrusion on the screen surface provided by this system strengthens the presence of the object and remote person. Keywords: Telepresence system Social telepresence Video-mediated communication Elastic display 1 Introduction Telepresence is a technology that allows a person to feel as if they were present at a place other than their true location [1 5]. One of the popular applications of telepresence is a video conferencing system. However, most of them are designed for flat surfaces where the perception of depth is lost. This decreases the presence of a remote person [6]. Some studies have tried to express a spatial three-dimensional effect of a twodimensional video with a curved or deformable display surface. For example, Livemask is a surrogate system with a face-shaped screen [7], and Interactive Spatial Copy Wall is a system, which represents a three-dimensional shape of a remote person with hundreds of movable pipes [8]. Furthermore, shape change is increasingly used in physical user interfaces, both as input and output [9 12]. We focused on the idea of representing a remote person with a flexible and deformable surface. We applied the elastic displays for a telepresence system as an output interface, which represents depth information about the video. The elastic displays are proposed for a new-generation input interface [13 16]. It is attained by a stretch cloth Springer Nature Switzerland AG 2018 H. Egi et al. (Eds.): CollabTech 2018, LNCS 11000, pp ,

2 PopObject: A Robotic Screen for Embodying Video-Mediated Object Presentations 201 Fig. 1. Snapshot of our deformable screen and offers new ways to interact with multi-dimensional data by using the deformation of the surface [17]. By applying an elastic display, we can build the deformable screen, which has a smooth surface and can express various shapes. Additionally, we attempted to verify the effectiveness of this simplified 3D surface for a telepresence system. Then, we conducted experiments in order to confirm an effect of the developed system as a telepresence system. 2 System The Snapshot of our deformable screen is shown in Fig. 1, and the constitution of the developed system is shown in Fig. 2. This system makes the object on the projected video appear protruded with its deformable surface. There was a projection screen, which was made of a stretch cloth and was flexibly deformed by pushing. The extruder was set behind the screen. If the object on the projected video moved forward, the extruder also moved forward and pushed the screen to the front. The screen was deformed along a shape of the extruder. This deformation of the screen adds the perception of the depth and a spatial threedimensional effect to the video. A shape of the extruder imitates that of the object. Figure 3 shows some examples of various objects and the extruders which correspond to each objects. Linear positioning tables that were set behind the screen moved to the extruder. The location of the extruder was synchronized with that of the object on the projected video.

3 202 K. Kushida and H. Nakanishi Linear motion table device Projector projects remote video on flexible screen Flexible screen deformed by extruder and expresses threedimensional effect Extruder Fig. 2. Mechanism of our deformable screen (a) Ball (b) Stuffed animal Fig. 3. The object (left) and the extruder corresponding to the object (right) 3 Experiment 1 At first, the experiment 1 was done as a preliminary experiment. We estimated a quality of the developed system. 3.1 Conditions In the experiment 1, we compared the following two conditions: Flat screen condition: The video of the remote explainer was projected on the flat screen (Fig. 4(a)). Deformable screen condition: The video of the remote explainer was projected on the screen and the screen was deformed along with the video (Fig. 4(b)).

4 PopObject: A Robotic Screen for Embodying Video-Mediated Object Presentations 203 (a)flat screen condition (b) Deformable screen condition Fig. 4. Conditions of the experiment Setup In this paper, we focus on the effect of the deformable screen on social telepresence, and we developed the one way system. Figure 5 shows the setup of the experiment. A vertical display monitor situated along one side of a desk provides a nearly identical image of the remote side. 3.3 Task We contrived a situation where the remote person explains while showing a ball or a stuffed animal as the task. In the ball task, the experimenter held out the ball and had a simple conversation with the subject. In the stuffed animal task, the experimenter held out the stuffed animal. In all conditions the presentation began and ended with a greeting. To conduct a controlled experiment, we offered the same conversation time. 3.4 Questionnaire We conducted a questionnaire after the experiment. The questionnaire included several statements and asked the extent to which the statements matched the impression that

5 204 K. Kushida and H. Nakanishi Subject (a) Local Side Web camera and kinect Explainer Object (b) Remote Side Fig. 5. Setup of the experiment 1 the participant had. The questionnaire asked five questions shown in Fig. 6. The subjects answered the questionnaires after they experienced both conditions. Q1, Q2 and Q3 check the quality of the presentation. Q5 correspond to presence of the object. Q4 correspond to presence of the remote person. All the statements were rated on a 7-point Likert scale where 1 = strongly disagree, 4 = neutral, and 7 = strongly agree. 3.5 Results and Discussion We compared two conditions by the within-subjects experiment. Eight subjects consisting of seven males and one females participated with the ball task. Also Eight subjects consisting of six males and two females with the stuffed animal task. The participants were undergraduate students whose ages ranged from 18 to 24 years. They

6 PopObject: A Robotic Screen for Embodying Video-Mediated Object Presentations Q1. The video was sufficiently clear. Q2. The audio was sufficiently clear. Ball Q3. The presentation was intelligible. Q4. I felt as if I were viewing the stuffed animal in the same room. Q5. I felt as if I were viewing the explainer in the same room. Q1. The video was sufficiently clear. Stuffed animal Q2. The audio was sufficiently clear. Q3. The presentation was intelligible. Q4. I felt as if I were viewing the stuffed animal in the same room. Q5. I felt as if I were viewing the explainer in the same room. Flat screen condition Deformable screen condition p<.01 p<.05 p<.1 Fig. 6. Results of the experiment 1

7 206 K. Kushida and H. Nakanishi were recruited via a part-time job search site, and paid about 20 dollars. We took counter balance to not produce an influence by the order of conditions. The result of the experiment is shown in Fig. 6, where each box represents the mean scores of the questionnaire, and each bar represents the standard error of the mean. We found no significant difference in Q1, Q2 and Q3. We found a significant difference in Q5 (ball task: p <.05, stuffed animal task: p <.01). We also found a significant difference in Q4 (stuffed animal task: p <.1). The subjects marked higher scores in the deformable screen condition than the flat screen condition in Q5, the questions regarding presence of the object. Since the same tendency was shown in both the ball task and the stuffed animal task, it was found that the effect of the deformation screen was obtained regardless of the complexity of the shape of the extruder. Additionally, The Q5 shows that presence of the remote person was also stronger in the deformable screen condition than other conditions. It suggested that the developed system enhances the social telepresence. 4 Experiment 2 We conducted experiments 2 in order to confirm an effect of the developed system as a telepresence system. We expected that a spatial three-dimensional feeling expressed by the deformable screen could enhance the presence of the remote person and object. We made the following hypotheses. Hypothesis 1: a spatial three-dimensional feeling expressed by the deformable screen enhance the presence of the object. Hypothesis 2: a spatial three-dimensional feeling expressed by the deformable screen enhance the presence of the remote person. 4.1 Conditions In the experiment, we compared the flat screen condition, deformable screen condition and Liquid crystal display condition. Liquid crystal display condition: The video of the remote explainer was exhibited on the liquid crystal display monitor like general videoconferencing systems (Fig. 7 (c)). The liquid crystal display condition was added in order to compare our system with the general videoconferencing systems. Liquid crystal displays have an advantage of video quality over projection screens. We expected that the deformable screen is effective to express the presence in despite of lower quality of the video.

8 PopObject: A Robotic Screen for Embodying Video-Mediated Object Presentations 207 (a)flat screen condition Deformation (b) Deformable screen condition (c)liquid crystal display condition Fig. 7. Conditions of the experiment 4.2 Setup We developed the one way system as with the experiment 1. Figure 9 shows the setup of the experiment. A vertical display monitor situated along one side of a desk provides a nearly identical image of the remote side. 4.3 Task We contrived a situation where the remote person explains an animal while showing a stuffed animal as the task. In all conditions, the experimenter held out the stuffed animal and had a simple conversation with the subject. To conduct a controlled experiment, we offered the same conversation time. In the deformable screen condition, the screen protruded along to the experimenter s arm and the stuffed animal with movement of the experimenter (Fig. 8).

9 208 K. Kushida and H. Nakanishi Area of the protrusion (a) Remote video (b) Extruder Fig. 8. The remote video and the extruder of the experiment Subject (a) Local side Web camera and kinect Explainer Object (b) Remote side Fig. 9. Setup of the experiment 2 (length unit: centimeters)

10 PopObject: A Robotic Screen for Embodying Video-Mediated Object Presentations Questionnaire We conducted a questionnaire after the experiment. The questionnaire included several statements and asked the extent to which the statements matched the impression that the participant had. The subjects answered the questionnaires after they experienced all three conditions. The questionnaire asked eight questions shown in Fig. 10. Q1, Q2, Q3 and Q4 check the quality of the presentation. Q5 and Q6 correspond to presence of the object. Q7 and Q8 correspond to presence of the remote person. All the statements were rated on a 7-point Likert scale where 1 = strongly disagree, 4 = neutral, and 7 = strongly agree. 4.5 Results and Discussion To clarify the hypotheses we compared three conditions by the within-subjects experiment. Fourteen subjects consisting of eight males and six females participated. The participants were undergraduate students whose ages ranged from 18 to 24 years. They were recruited via a part-time job search site, and paid about 20 dollars. We took counter balance to not produce an influence by the order of conditions. The result of the experiment is shown in Fig. 10, where each box represents the mean scores of the questionnaire, and each bar represents the standard error of the mean. The figure compares the conditions by a one-way ANOVA, followed by Bonferroni correction. We found no significant difference in Q1, Q2, Q3, Q4 and Q8. We found a significant difference in Q5 (F(2, 26) = 12.82, p <.01) and Q6 (F(2, 26) = 13.26, p <.01). Multiple comparisons showed that presence of the object was significantly stronger in the deformable screen condition than the flat screen condition and liquid crystal display condition (p <.05). We also found a significant difference in Q7 (F(2, 26) = 8.000, p <.01). Multiple comparisons showed that presence of the remote person was significantly stronger with the deformable screen condition than the flat screen condition and liquid crystal display condition (p <.05). Contrary to expectation, we found no significant differences in Q1, the question regarding quality of the video, because eight subjects did not perceive a difference in the display method. Six subjects recognized the difference between the flat screen condition and the liquid crystal display condition. However, the subjects marked higher scores in the deformable screen condition than the liquid crystal display condition in Q5 and Q6, the questions regarding presence of the object. It suggested that the developed system has a potential compared to the general videoconferencing systems in despite of lower quality of the video. Additionally, the Q7 shows that presence of the remote person was also stronger in the deformable screen condition than other conditions. According to the results of the interviews, we found that there are individual differences in the perception of depth information in the deformable screen condition. One subject believed that there was a three-dimensional feeling on the entire screen in despite of the fact that we deformed only part of the screen. On the other hand, two subjects did not perceive a deformation of the screen. Eleven subjects recognized that only the stuffed animals protruded and the experimenter does not. During the task, these subjects paid attention to the experimenter s face. The three-dimensional feeling of the experimenter s arms were

11 210 K. Kushida and H. Nakanishi Q1. The video was sufficiently clear Q2. The audio was sufficiently clear. Q3. The presentation was intelligible. Q4. The shape of the stuffed animal was easy to understand. Q5. I felt as if I were able to touch the stuffed animal. Q6. I felt as if I were viewing the explainer in the same room. Q7. I felt as if the experimenter were facing me. Q8. I felt as if I were viewing the stuffed animal in the same room. p<.05 Flat screen condition Deformable screen condition Liquid crystal display condition Fig. 10. Results of the experiment 2

12 PopObject: A Robotic Screen for Embodying Video-Mediated Object Presentations 211 frequently ignored. Another factor was caused by the weak point of this deformable screen that cannot express a detail of the complex shapes. The deformable screen could not express the shape of the remote person s arms clearly. However, many of the subjects felt presence of the remote person although they did not percept the threedimensional feeling of the remote person. In this case, the presence of the object enhanced feeling to share the object with remote person, and it enhanced presence of the remote person. Future studies are needed in order to compare the deformable screen and an object sharing system. 5 Conclusion In this study, we developed a telepresence system which has a stretch projection screen and a push-out mechanism. The screen deforms in synchronization with the move of the objects on the projected video. This deformation highlights a specific part of a remote video by a protrusion on the display surface. This protrusion is aiming to express the depth information of the remote video so we expected that it enhances the remote conversation partner s presence. We conducted experiments in order to confirm an effect of the developed system as a telepresence system. The results of the experiment showed that the protrusion on the screen surface provided by this system strengthens the presence of the object and remote person. Additionally, It was suggested that this system is more valid than the liquid crystal display in despite of lower quality of the video. Acknowledgments. This work was supported by JSPS KAKENHI Grant Numbers JP , Telecommunication Advancement Foundation, and Tateishi Science and technology Foundation. References 1. De Greef, P., Ijsselsteijn, W.A.: Social presence in a home tele-application. CyberPsychology Behav. 4(2), (2001) 2. Nakanishi, H., Kato, K., Ishiguro, H.: Zoom cameras and movable displays enhance social telepresence. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp ACM (2011) 3. Onishi, Y., Tanaka, K., Nakanishi, H.: Embodiment of video-mediated communication enhances social telepresence. In: Proceedings of HAI 2016, pp (2016) 4. Wesugi, S., Miwa, Y.: Facilitating interconnectedness between body and space for fullbodied presence-utilization of Lazy Susan video projection communication system. In: Exploring the Sense of Presence 2004, pp (2004) 5. Leithinger, D., Follmer, S., Olwal, A., Ishii, H.: Physical telepresence: shape capture and display for embodied, computer-mediated remote collaboration. In: Proceedings of UIST 2014, pp (2014) 6. Prussog, A., Mühlbach, L., Böcker, M.: Telepresence in videocommunications. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 38, no. 3, pp (1994)

13 212 K. Kushida and H. Nakanishi 7. Misawa, K., Ishiguro, Y., Rekimoto, J.: Livemask: a telepresence surrogate system with a face-shaped screen for supporting nonverbal communication. Inf. Media Technol. 8(2), (2013) 8. Wesugi, S., Ishikawa, K., Suzuki, N., Miwa, Y.: Interactive spatial copy wall for embodied interaction in a virtual co-existing space. In: th IEEE International Workshop on Robot and Human Interactive Communication, ROMAN 2004, pp IEEE (2004) 9. Follmer, S., Leithinger, D., Olwal, A., Hogge, A., Ishii, H.: inform: dynamic physical affordances and constraints through shape and object actuation. In: Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology, pp (2013) 10. Holman, D., Vertegaal, R.: Organic user interfaces: designing computers in any way, shape, or form. Commun. ACM 51(6), (2008) 11. Rasmussen, M.K., Pedersen, E.W., Petersen, M.G., Hornbæk, K.: Shape-changing interfaces: a review of the design space and open research questions. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp ACM (2012) 12. Sahoo, D.R., Hornbæk, K., Subramanian, S.: TableHop: an actuated fabric display using transparent electrodes. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pp ACM (2016) 13. Dand, D., Hemsley, R.: Obake: interactions on a 2.5 D elastic display. In: Proceedings of the Adjunct Publication of the 26th Annual ACM Symposium on User Interface Software and Technology, pp ACM (2013) 14. Peschke, J., Göbel, F., Gründer, T., Keck, M., Kammer, D., Groh, R.: DepthTouch: an elastic surface for tangible computing. In: Proceedings of the International Working Conference on Advanced Visual Interfaces, pp ACM (2012) 15. Troiano, G.M., Pedersen, E.W., Hornbæk, K.: User-defined gestures for elastic, deformable displays. In: Proceedings of the 2014 International Working Conference on Advanced Visual Interfaces, pp ACM (2014) 16. Brave, S., Ishii, H., Dahley, A.: Tangible interfaces for remote collaboration and communication. In: Proceedings of the 1998 ACM Conference on Computer Supported Cooperative Work, pp ACM (1998) 17. Müller, M., Knöfel, A., Gründer, T., Franke, I., Groh, R.: FlexiWall: exploring layered data with elastic displays. In: Proceedings of the Ninth ACM International Conference on Interactive Tabletops and Surfaces, pp ACM (2014)