Daily Life Support Experiment at Ubiquitous Computing Home

Transcription

1 Daily Life Support Experiment at Ubiquitous Computing Home Michihiko Minoh Tatsuya Yamazaki Abstract We have constructed a real-life test bed, called the ``Ubiquitous Computing Home (UHC)," for home context-aware service experiments. Since a family consists of persons of various ages and the household is a place where families are relaxed, the strategy of developing the test bed is completely different from that for office systems. Our ideas for constructing UCH are: (1) each function element of an appliance is independently connected to the Home Network. (2) UHC is unconscious robot and interface robot is a conscious robot, metaphorically identified a Mother-Child. (3) Natural Language is used for universal interface at home. In this paper, we discuss the design and implementation of the test bed and several context-aware services that support human daily life. Keywords: Ubiquitous computing, Home application, Test bed, Daily life support, motherchild metaphor. 1 Introduction The networking of all sorts of devices is creating a new lifestyle in the 21st century. Particularly, a home is basic living place for humans and human life. Although until now, there is no systematic approach to introduce ICT to the home, networked appliances and home network are gradually being introduced. In near future, support of human daily life at home will become important. This is the motivation to construct a real-life test bed, called the ``Ubiquitous Computing Home (UCH)," for developing daily human life support services. Since a family consists of persons of various ages and the household is a place where families are relaxed, the strategy of developing the test bed is completely different from that for office systems. For the home applications, there are two important issues, the choice of platform, and the choice of interface. For the former issue, we design a distributed cooperative function platform called UKARI core, which is named after our project name UKARI: Universal Knowledgeable Architecture for Real-LIfe appliances. For the latter issue, we introduce a cute robot called Phyno, which can interact with the user via natural language. This would be a universal interface for person in all the ages. There are several researches that are related to ours. The real-life living space test beds are sometimes called a smart home or sensorembedded house. The Aware Home project[1] is a noteworthy and successful one. In this project, they built a three-story, 5040-squarefoot home that functions as a living laboratory for interdisciplinary design, development and evaluation. Extended from Robotic Room 2[2], the Sensing Room was constructed at the University of Tokyo[3]. The sensing room of EasyLiving project at Microsoft Research[4] is similar to Sensing Room. In EasyLiving project, they aimed at developing a proto type architecture and technologies for building intelligent environments. The University of Florida's Mobile and Pervasive Computing Laboratory is developing programmable pervasive spaces in which a smart space exists as both a runtime environment and a software library[5]. The University of Sherbrooke of Canada constructed DOMUS laboratory, which is a new research pole on cognitive assistance in smarts homes

2 and mobile computing[6]. Active Home is a demonstration space of Information and Communications University of Korea and several experiments of context-aware services according to human behaviors are carried out[7]. UbiHome is another smart space in Korea, where automated control of lights and monitors according to user's situations are demonstrated [8]. In these researches, one clear purpose such as how to get the data from sensors, to support elderly persons and architecture for technical experiments etc. is set up. Different from these, our purpose is to construct a real-life test bed, called UCH for supporting human daily life totally[9], in which a family can live. This paper is organized as follows. In section 2, the concept of Mother-Child metaphor is discussed. In section 3, sensing and utilizing sensing data are discussed from both technical and privacy view points. The facility of unconscious robot UCH and visible robot Phyno are explained in section 4 and 5, respectively. In section 6, the context aware services we have developed are presented. The real living life experiment is described in section 7 and the conclusion is in section 8. 2 Mother-Child metaphor One of the basic ideas of the UKARI project is to introduce the two types of robots, i.e. unconscious type home robot and visual type interface robot, which we call Mother-Child metaphor (Figure 1). Namely, the UCH, highly networked and sensor-embedded home, is the unconscious type robot which is invisible to the family and autonomously controls appliances on the network in accordance with various kinds of sensor information. The visible type robot that actually exists and the family recognizes as it is at a glance is needed to be the universal interface for all the members of the family. Figure 1 Mother Child Metaphor 2.1 UCH as Mother metaphor It is becoming possible to connect home appliances to home network. Since the home appliances are designed to be used independently, there is little demand to connect them to the home network because one of the advantages of the home network is to share appliances. Introducing the home network at home has to have a great advantage. Generally, an appliance has several functions. For example, Television set has a display to present images, two speakers to generate speech/sound and a tuner to select a specific channel. If these functions are independent one another and connected independently to the home network, a new appliances is virtually constructed that consists of, for example, several functions from appliance A, some from appliance B and the others from appliance C. (See Figure 2) Figure 2 UKARI core and a virtual appliance

3 To realize it, we have developed the middle ware named UKARI core that enables an appliance to connect its functions independently to the network. In this case, the problem how to control the functions in the appliance that has its own controlling mechanism because the appliance has to be used normally whether it is connected to the network or not. The UKARI core does not work when the host controlling mechanism is in motion. Sensors are considered to be a device with only one function. Using the UKARI core, many appliances and sensors in the household are connected to the home network. In a sense, the house becomes a robot in which families live. This robot is called unconscious robot because the family is not conscious of it and the whole figure of it can not be seen. Still, the robot gently watches the family and finds out whether it can help them or not. This is similar to that of mother who watches her child. 2.2 Visible robot as Child metaphor On the contrary, a visible robot is introduced to request services at home. Since the family consists of members in various ages, universal interface is absolutely necessary. Natural language interface is one of the most preferable universal interfaces. Therefore, we set up an interface robot called Phyno in each room of the house that can communicate with the family via natural language. The ability of recognizing the speech of Phyno is about that of 3 years old child, hence the figure of Phyno is designed to be similar to 3 years infant (see Figure 3). However, he knows a lot more about the appliances at home and he can control almost all the appliances at home. In this sense, he has more knowledge than the family has. This situation makes the communication between Phyno and family active. Figure 3 Universal interface robot Phyno 3 Sensing human behavior for context aware services To provide context aware service, the personal identification i.e. ``who the user is" is necessary because even in a family the behavior of each member is so different from one another. The context we consider here is the location of a member of the family and his/her log of the information and operation to the appliances. In this case, the logs for each member of the family have to be collected independently. Hence, we need the technique for personal identification at home. The personal identification used at home has to be as natural as possible. In this context, face recognition or voiceprint recognition is preferable compared with using fingerprint or vein patterns. At the Ubiquitous Computing Home, a personal identification method using face recognition has been installed in the visibletype robot, so that the resident can be recognized by the networked robots when he/she speaks to the visible-type robot. Sensor technologies are also essential to obtaining the user context. An active-type RFID tag system is used to identify a resident. The location information of the user is also important as the context. Floor pressure and vibration sensors are used to collect the location information of the residents. The camera can observe human behavior and the microphone can observe human voice. Collecting all the information from all the sensors at home, the unconscious robot UCH can detect the context of the user. Based on the context, the system invokes, what we call, context aware services for supporting families daily life. A house is the place where families live. It has to be occupied and be kept private. Introducing sensors at home has to be considered carefully. There are various kinds of sensors that can be installed at home; cameras, microphones, infrared sensors, RFID tags, on-off switch sensors, etc. Among them, the camera catches behavior of humans and the microphone picks up voices, both of which belong to the personal information. If such data happen to be watched by others, the privacy will be lost. Hence, the system installed at home has to protect the collected data and restrict users to watch them.

4 4 Facility of UCH The Ubiquitous Computing Home (UCH) is located at the Keihanna Human Info- Communication Research Center building of National Institute of Information and Communication Technology of Japan (NICT). It is an ICT housing test bed for developing useful new services of the home that connects devices, sensors and appliances by means of a home network. The layout of the experimental UCH is shown in Figure 4, which has a living room, diningkitchen, study room, bedroom, washroom and bathroom. In addition, a Japanese-style room has been built as a simulated living space for remotely living family members, such as a grandmother and/or a grandfather. Between them, a computer room called the Network Operating Center (NOC) is located. sensors have been attached to every door of the house and of the shelves, and every drawer. Floor pressure sensors installed throughout the flooring contain 18 cm by 18 cm binary detection units and are used to track the family or detect furniture positions. Infra-red sensors installed at the top of the entrance door in each room, and at foot positions in the kitchen and in the corridor detect infra-red sources literally, so can be used to detect the movement of the family. The RFID systems of 13.56MHz and 2.45 GHz bands are passive-type and the system of 315MHz band is active-type. The active-type system scanners located over the ceiling of each room detect the RFID tags held by the family. Four accelerometers or vibration sensors are attached to the back of the bedroom floor in four corners. Although the roles of the accelerometers are similar to the floor pressure sensors, they are more promising to detect human behavior owing to their higher precision compared the floor pressure sensors. As the actuators, there are displays (plasma display panels and liquid crystal displays), speakers and networked consumer appliances such as a refrigerator and a washing machine. Figure 5 shows a view of the living room of the UCH. Figure 4 Layout of UCH Above the ceiling of the UCH, there is a space in which three corridors called ``cat walks" are established. These are used for maintenance i.e. machine installation and cabling, etc. The floor of the UCH is the free access floor of 40 cm height. Figure 4 also presents the arrangement of sensors and actuators in the UCH. As the sensors, there are cameras and microphones in the ceiling, floor pressure sensors throughout the flooring, infra-red sensors at the top of the entrance door of each room and at foot positions in the kitchen. In the corridor, three RFID systems (13.56MHz, 315MHz, 2.45 GHz bands) and vibration sensors exist. 113 on-off switch Figure 5 View of living room in UCH 5 Facility of ``Phyno" Phyno has a camera, three microphones and a speaker. Its neck has three degrees of freedom (DOF), its arm and waist have one DOF, respectively. The lens of the Phyno s camera is a wideconversion lens to cope with the problem that the face size changes significantly depending on the distance between the camera and the human

5 face. The method in [10] was applied to calibrate the image distortion caused by the wide-conversion lens[11]. Consequently, the whole of human face can be captured both at a short distance(20 cm) and at a long distance(60 cm) as shown in Figure 6. data and presented to the speaker closest to the user. The service scenario is shown in Figure 7. Figure 7 Intelligent message presentation Figure 6 Captured faces by Phyno A decomposed eigenface method is used for the face recognition. The method realizes robust recognition under various lighting conditions. Refer to [12] for the details of the face recognition system. 6 Context Aware Services At the UCH, many appliances and many sensors are connected to the home network. Using the sensors, the unconscious robot can collect the context information of each member of the family. The context we use for the context aware services is the location of a member of the family and his/her log of the information and operation to the appliances. The position and personal identification can be easily obtained by the active-type RFID tag system, though other sensors such as floor sensors, the cameras and the microphones can also provide the information used to calculate the position and personal identification. We have implemented several context-aware services using the RFID tag system. Here they are briefly explained. These services are actually used in the real living experiment described in the next section. (1) Intelligent message presentation This is a collaboration service of a laundry, speakers and the RFID system. The laundry can transmit its status information such as ``washing" or ``drying finish" in the text format to the home network. In the context-aware service, this message is converted into the voice (2) Wake up service This is a collaboration service of a sleeping sensor on the bed, an electric clock and Phyno. When the time to wake up has come, the sleeping sensor checks the depth of sleep of the user and Phyno plays music that is suitable to the depth and that is preferable to the user. The context in this service is the depth of the sleep that is sensed by the sleeping sensor. (3) Have you got everything? This service is a collaboration service of a floor sensor, Phyno and the RFID system. When a user of this service appears in the hall, the floor sensor detects it and inform Phyno to speak to the user that he/she should present his/her baggage in front of the RFID tag reader. If the unconscious robot knows what items the user should have before going, it can let the user know what item is missing. (4) Music follows wherever you go This service is a collaboration service of speakers, music player and the 315MHz band RFID system. When the RFID system detects the movement of the user who is listening to the music, the system tracks the user and using the nearest speaker available of the user, the music continues. (5) Recommendation of TV program This service is a collaboration service of TV set, history server and the RFID system. The history of the user which TV program the user watches is stored in the database. When the user comes and turns on the TV, the system retrieve the database, and recommend a TV program that may be preferable to the user.

6 7 Real Living Experiments at Ubiquitous Computing Home We are conducting a real-life experiment in which a family stays at the UCH for about two weeks and they use the visible-type robot as a universal user interface to utilize the contextaware services. Purposes of this experiment are: How people feel when living in such a sensor based home. How people behave when interacting with Phyno. Whether the context aware services we have developed are useful or not. So far, five families lived in the UCH. All sensor data for all families and for all days are stored. We have not analyzed the data in detail yet. After the experiment, we interviewed some of the people who lived in the UCH. The following are a summary of the experimental results from their viewpoints. 1. Many subjects said that at first they were aware of the sensors in the rooms but after 3 days they got used to them. Some parents said that they found very interesting to watch their children behave while they were out of home. 2. For the interface robot Phyno, they tried to find the personality of it. However, Phyno is located each room, so they are confused. The question arises: how do we locate Phyno in our mind? Is it a human, just a pet or a metal robot? 3. They said that it was better to be watched by Phyno than the camera on the ceiling. It may be a cultural matter. 4. They tried to communicate with Phyno, because it can control almost all the appliances at home. When they can communicate with Phyno, they can live a comfortable life. They change the way to speak to Phyno so much so that Phyno can recognize their speech. 5. When the subjects spoke to Phyno and it can not recognize the speech, Phyno can not do anything, which cause the subjects feeling of frustration. To design an interface robot, we have to consider the action when such problem occurs. Now, we introduce a strategy, according to which Phyno repeats the speech that it has just heard. This strategy did work to a certain extent. 8 Conclusion In this paper, we discussed the daily life support experiment at UCH that is equipped many sensors and the universal interface robot Phyno here and there. We conducted several real life experiments and got several important results: living in such a sensor equipped home in future may be comfortable. To use sensors causes a trade-off problem between privacy and public interest and convenience. For this problem, our strategy is as follows. The data is obtained, by the person, to the person, and for the person who is observed. This aims at controlling all the personal data by the person who is observed. We have to continue to develop what services are useful and to extend real life experiment to the people who are not interested in ICT and Robot technology in future. Acknowledgements The authors would like to thank all contributors in the UKARI project for their support in constructing the Ubiquitous Computing Home. References [1] Kidd, Cory D., Robert J. Orr, Gregory D. Abowd, Christopher G. Atkeson, Irfan A. Essa, Blair MacIntyre, Elizabeth Mynatt, Thad E. Starner and Wendy Newstetter, ``The Aware Home: A Living Laboratory for Ubiquitous Computing Research," Proc. of the Second International Workshop on Cooperative Buildings - CoBuild'99, [2] T. Mori et al., ``Accumulation and Summalization of Human Daily Action Data in One-room-type Sensing System, " Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems 2001, pp , [3] Taketoshi Mori, Hiroshi Noguchi, Aritoki Takada, Tomomasa Sato, ``Sensing Room: Distributed Sensor Environment for Measurement of Human Daily Behavior," Proc. of First International Workshop on Networked Sensing Systems (INSS2004), pp.40-43, 2004.

7 [4] Brumitt, B., Meyers, B., Krumm, J., Kern, A., and Shafer, S. ``EasyLiving: Technologies for Intelligent Environments," Hand held and Ubiquitous Computing 2000, in LNCS 1927, Springer-Verlag, Sept [5] A. Helal, W. Mann, H. Elzabadani, J. King, Y. Kaddourah and E. Jansen, ``Gator Tech Smart House: A Programmable Pervasive Space," IEEE Computer magazine, pp.64-74, March [6] Pigot H., Lefebvre B., Meunier J.G., Kerherve' B., Mayers A., Giroux S., ``The role of intelligent habitats in upholding elders in residence," 5th international conference on Simulations in Biomedicine, Slovenia, April [7] Jae-Seon Lee, Kyoung-Shin Park, Min-Soo Hahn, ``WindowActive: An Interactive House Window On Demand," 1st Korea- Japan Joint Workshop on Ubiquitous Computing and Networking Systems (UbiCNS 2005), pp , June [8] Yoosoo Oh and Woontack Woo, ``A unified Application Service Model for ubihome by Exploiting Intelligent Context-Awareness," Proc. of Second Intern. Symp. on Ubiquitous Computing Systems (UCS2004), pp , Tokyo, [9] Tatsuya Yamazaki, ``Ubiquitous Home: Real-life Testbed for Home Context-Aware Service," Proc. Tridentcom 2005 (the First International Conference on Testbeds and Research Infrastructures for the DEvelopment of NeTworks and COMmunities), pp , Trento, Italy, [10] Z. Zhang, ``A flexible new technique for camera calibration," IEEE Trans. Pattern Analysis and Machine Intelligence, vol.22(11), pp , [11] Junji Satake, Masaki Chikama, Fumihiko Sakagami, Takeshi Shakunaga, Hirotada Ueda, ``Prototyping of Dialog Interface Robot combining Face Recognition and Speech Recognition in Ubiquuitous Environment," IPSJ Computer Vision and Image Media (CVIM), Vol.105, No.1, pp.1-8, Sept (in Japanese). [12] Fumihiko Sakaue, Tsuyoshi Migita, Takeshi Shakunaga, Junji Satake, Masaki Chikama, Hirotada Ueda, ``Face Discrimination System for Dialog Interface Robot in Ubiquitous Environments," IEICE Pattern Recognition and Media Understanding (PRMU), No.PRMU , pp.27-32, Oct (in Japanese).