9/19/15. Case Studies in Design Informatics Jon Oberlander. Lecture 3: State-of-the-art in the design of domestic and long-term robots

Transcription

1 Care- o- bot Case Studies in Design Informatics Jon Oberlander Lecture 3: State-of-the-art in the design of domestic and long-term robots Course Timetable Structure of lecture Week Topic Mon Mon Thu 1 HRI Intro (JO) <No class> 2 HRI Designing a robot (JO) Tutorial State-of-the-art (JO) 3 HRI Towards JAMES (JO) Tutorial JAMES (JO) 4 HRI HRI vs HDI (JO) Tutorial Animals A1 5 Eval Usability evaluation (JO) Tutorial Usability evaluation (JO) Submit 16:00 Thu 6 Data Personal data (DMR) Tutorial Personal data (DMR) A2-draft 7 Data Personal data (DMR) Tutorial* Personal data (DMR) 8 Data Personal data (DMR) Tutorial Personal data (DMR) A2 9 New Reflection (JO) Tutorial ADI 1 1. (Yet Another) Angle on social robots Hegel et al Designing a robot butler Reiser et al Programming vs robot interaction design Glas et al Acceptability of domestic robots Young et al Evaluating long term human-robot interaction Leite et al Lessons 10 New ADI 2 Tutorial ADI 3 11 New ADI 4 (Tutorial) ADI 5 A

2 Hegel et al Designing a robot butler: Reiser et al (Cogniron) Frank Hegel, Claudia Muhl, Bri;a Wrede, Mar=na Hielscher- Fastabend, Gerhard Sagerer (2009) Understanding Social Robots nd Intl Conf on Advances in Computer- Human Interac=on! Social robot = robot + social interface! A social interface encloses all the designed features by which a user judges the robot as having social quali=es.! In principle, it is a metaphor for people to interact naturally with robots.! This is an analogy to the desktop metaphor on computers where people treat the things in the graphical user interface like in their real world - due to the metaphor they have an idea on how it works. Ulrich Reiser, Chris=an Conne;e, Jan Fischer, Jens Kubacki, Alexander Bubeck, Florian Weisshardt, Theo Jacobs, Christopher Parlitz, Mar=n Haegele, Alexander Verl (2009) Care- O- bot 3 - Crea=ng a product vision for service robot applica=ons by integra=ng design and technology. The 2009 IEEE/RSJ Interna=onal Conference on Intelligent Robots and Systems October 11-15, 2009 St. Louis, USA! Design of Care- o- bot 3: Goal: a unique and iconic design for a service robot depic=ng an innova=ve product percep=on away from a humanoid approach. Intended to perform fetch- and- carry tasks and to handle the infrastructure of a household, e.g. cupboard, dishwasher, fridge, etc., [So,] safe manipula=on is needed Quoting from Hegel et al Care- o- bot Process and fundamental concepts Fig. 2. Left: First design sketch. Right: First technical rendering.! To extract necessary func=onality, define: roles (Butler, Info- Terminal, A;rac=on,...) tasks (Lay a table, Serve drinks, Fetch- and- Carry)! evaluate: state- of- the- art robot technology; experience on previous projects; domes=c constraints on size and weight.! Fundamental concept: two sides working side = back, away from the user; manipulators and sensors - can't be hidden; need environmental access. serving side = front; no fears of mechanical parts - smooth surfaces, likeable appearance.! Secondary concept: tray passing objects directly from human to robot via a robot's gripper was not sa=sfying - =ming of object release too hard. so tray is an interface between robot and human for the passing of objects. add: a touchscreen - for tradi=onal human- computer interac=on. tray can be retracted so that the robot is compact in stand- by.! Finally, torso is flexible: allows simple gestures like bowing or nodding - a butler style of communica=on

3 Care- o- bot Hardware and evalua=on Fig. 9. The two sides concept: The robot is commanded via the touchscreen to fetch a drink and serve it to the user. During the interaction, the working side of the robot with its arm points away from the user.! Close connec=on between soh and hardware development.! Hardware setup includes altogether 28 DOF: mobility base, torso, manipulator, tray, and sensor carrier with sensors.! Component evalua=on is feasible, as usual.! Whole system evalua=on is trickier.! Use case evalua=on: serving drinks robots offers a selec=on of drinks to the user user chooses a drink on the touch- screen robot drives to a bar and tries to detect the chosen drink. If the bo;le is succesfully detected, it is grasped and placed on the tray. robot proceeds towards a door which it has to open and pass. Behind the door the robot grabs a cup to put on top of the bo;le. robot drives back to the user and serves bo;le and cup via the tray Care- o- bot Programming vs robot interac=on design: Glas et al. 2012! Evaluation TABLE III Dylan F. Glas, Satoru Satake, Takayuki Kanda, Norihiro Hagita (2012) An Interac=on Design Framework for Social Robots. In Proc. Robo=cs: Science and Systems, 7, p89ff. SUCCESS RATES OF COMPONENTS AND ROBOT; N s GIVES THE ABSOLUTE NUMBER OF SUCCESSES; R s GIVES THE SUCCESS RATE OF THE SINGLE COMPONENTS AND THE FULL SCENARIO IN PERCENT Hardware object bottle door cup done detect grasp open place task drive N s R s 86.7% 100% 100% 73.3% 93.9% 86.7% 40%! Social robot development must deal with the conven=onal challenges of robo=cs e.g. robot localiza=on and mo=on planning,! and with dis=nc=ve new challenges: e.g. new kinds of sensory- informa=on processing, dialog management, the applica=on of empirical design knowledge in interac=on. e.g. maintaining acceptable interpersonal distance, approaching people from a non- frontal direc=on controlling the dura=on and frequency of eye contact

4 Requires collabora=ve design! Programmers algorithms and sohware modules for informa=on- processing tasks e.g. human tracking, social group detec=on, gesture recogni=on, predic=on of human behavior, or dynamic path planning.! Non- programmers scrip=ng the robot's u;erances, choosing gestures, and structuring the sequence of the robot's ac=ons.! Development processed slowed if non- programmers require programmers at every step.! Proposed design framework a framework which uses clearly- defined layers of abstrac=on to allow parallel development. Tradi=onal dialogue management! State- based systems generate u;erances and recognize users' responses according to a state- transi=on model, like a flowchart. simple and intui=ve, easy to implement and ohen used in working systems.! Frame- based systems fit users' responses into pre- defined slots in *frames* to es=mate user goals. ohen used for telephone- based dialogue systems, e.g. for providing weather or transporta=on informa=on. Frame- based systems can handle more complex informa=on, but involve more effort for prepara=on of such frames of knowledge.! Plan- based, or agent- based, systems use a set of rules to change the internal states of an agent to navigate through conversa=on. These can handle the most complex interac=ons, but require very advanced natural- language processing and well defined sets of rules. This approach is ohen used in research but rarely used in working systems Dialogue in robots! In robo=cs, dialogue management has some=mes been studied while taking real- world difficul=es into considera=on.! the majority of such systems have been task- oriented, that is, aimed primarily at communica=ng commands or teaching informa=on to a robot! different from social dialogue, where the goal of the robot may be to entertain, interest, or persuade a customer. 15 Interac=on Design Framework: Division of roles! Programmer Hardware interfacing: Adding new sensors or actuators to the system will require work at the robot driver level to enable the new components to operate with the robot's control system. Data processing: New recogni=on techniques or machine learning algorithms will be necessary to help the robot understand the situa=on in its environment. Behavior development: Basic interac=ve robot behaviors need to be developed, e.g. a behavior for approaching a moving person in a socially- appropriate way, based on tracking informa=on from an external sensor network.! Interac=on Designer Dialogue genera=on: An interac=on designer will need to specify the robot's u;erances and gestures. To tune the robot's performance, a designer could adjust the speed of the robot's ac=ons or speech, or insert appropriate pauses. Interac=on flow design: a designer can create interac=on flows. [the] order [to] present informa=on, ask ques=ons, respond to a person's ac=ons. Non- dialogue elements could be used in these flows, such as driving to a new loca=on or approaching a customer. Content entry: It may also be necessary to enter large amounts of domain- specific content, such as items in a restaurant menu, details about products in a store, direc=ons to loca=ons in a shopping mall, or informa=on about seasonal events. require[s] specific domain Quoting knowledge from Glas et al

5 Interac=on Design Framework Interac=on Design Framework! Robot Driver Layer contains hardware- specific driver modules. supports abstract interfaces that hide minor differences between similar robots, e.g. different motor or joint configura2ons, or size differences! Informa=on Processing Layer contains sensing and actua=on modules. Sensing modules [help] recogni=on of environments and ac=vi=es in the real world. e.g. localiza2on, human tracking, face detec2on, speech recogni2on, and sound source localiza2on. Actua=on modules perform processing for tasks e.g. path planning or gaze following.! Behavior Layer The concept of a robot *behavior* as a combina=on of sensor processing and actua=on is used both in behavioral robo=cs - and in social robo=cs. e.g. guide behaviors incorpora2ng speech, gesture, and 2ming, or approach behaviors which react to a person's trajectory! Applica=on Layer where designers can develop social robot applica=ons. Interac=on Composer enables interac=on flows to be built by assembling behavior and decision blocks into sequences [with interrupts] resembling flowcharts. Figure 1. Four-layer robot control architecture Design Framework! Evalua=on: A lot more got done! e.g. programming robot to do 2 tasks; 2nd task was successfully programmed in 94% of cases with Interac=on Composer vs 19% without. Acceptability of domes=c robots: Young et al James E. Young, Richard Hawkins, Ehud Sharlin, Takeo Igarashi (2009) Toward Acceptable Domes=c Robots: Applying Insights from Social Psychology. Int J Soc Robot (2009) 1: ! unique barrier to... domes=c adop=on of robo=cs is an especially complex socializa=on process.! The robo=cs environment is far more complex than most already established consumer technology markets, and! the problems of technology acceptance are far more significant in a domes=c environment than in an industrial one. Figure 2. Screenshot of Interaction Composer

6 Domes=c robots! We define a domes=c robot to be a machine that a) is designed to work with individuals and groups in their personal and public spaces, b) has a dynamic spa=al presence in those spaces, and c) can *intelligently* interpret its environment and interact physically with it.! Our defini=on does not require robots to resemble humans, to be mobile, or to communicate using natural language. 21 Issues! Robots have an invasive physical presence and a unique interface paradigm: they ac=vely and physically share spaces with people and display a level of autonomy and intelligence. [Unlike the PC]! interac=ng with a robot is more like interac=ng with a living en=ty. The robot may move unexpectedly, users must follow its mo=on cues and physical state, and may not have direct access to orthodox interfaces such as a keyboard or display panel.! Thus, users of robo=c technology ohen have to learn new interac=on styles such as manipula=on through remote control devices or voice commands [but - social interface is the universal interface?]! Design methodology, such as appearance, ac=ons, and behavior, will have a large affect [sic] on how people perceive domes=c robots and the condi=on of owning one.! the communica=on paradigm... will be a crucial component in the robot's chances of acceptance into an applica=on environment. 22 Acceptance! The Model of Acceptance of Technology in Households (MATH) [Venkatesh] a domes=ca=on- of- technology framework that focuses on the home, developed around an extensive longitudinal study of the adop=on of PCs into over seven hundred households all across America, primarily concerning the factors that people cited for or against adop=on. e.g. 45% of those who claimed they intended to adopt the PC did so six months later, sugges=ng that fears may strongly overpower perceived gains. non- adopters primarily cited fears of technology obsolescence. Factors Affec=ng Acceptance! 1. Safety: Robots have an autonomous physical presence that, in a worst- case- scenario, can damage household objects or seriously injure and kill people. Robots provide a level of poten=al danger seldom experienced with other domes=c technologies in the past, and so this concern may be dispropor=onately important and may overshadow any and all other gains and benefits.! 2. Accessibility and usability: The capabili=es and complexity of robots raises serious accessibility concerns. Exis=ng technology fears such as lack of knowledge, usability, and behavioral control (already shown to have been a problem for PC adop=on), will escalate given the physical presence and dangers of robots. Other barriers include facili=es and space requirements within the home, financial prac=cality (affordability, maintenance, obsolescence) & legal barriers & regula=ons.! 3. Prac=cal benefits: People really care about the u=lity gains promised by robots, and the poten=al impact on their quality of life. Robots must not only be useful, but need to fit properly into the social structures of a given par=cular lifestyle. however, people may be overly dubious about the capabili=es of robots. (b) RIKEN RI-MAN

7 Factors Affec=ng Acceptance! 4. Fun: Direct fun (and secondary gains such as more free =me due to u=lity gains) is a very important considera=on for domes=c procurement decisions. Robot designers have already recognized this and have introduced robo=c technology sa=sfying this need. Further, companionship and comfort are basic human needs that robots may be able to meet. Perhaps, similar to the way games help drive PC technology, entertainment robots may serve as a catalyst for the en=re domain.! 5. Social pressures: Conflic=ng social pressures should be expected concerning domes=c robots. As robots become common we can expect the emergence of social pressures that will mo=vate adop=on, for example the mo=va=on for a family to appear to be *modern*. On the other hand, nega=ve pressures such as appearing lazy or wasteful can also be expected. Factors Affec=ng Acceptance! 6. Status gains: Being perceived as a cuxng- edge person or family, or being recognized as a knowlegable reference by neighbors or co- workers, has been important for adop=on of technologies in the past. It is unclear how this will relate to domes=c robots, but science fic=on and research hype has arguably created a fairly posi=ve and luxurious image of robots for people to consider. Designers can take advantage of this image.! 7. Social intelligence: There is a tendency for people to anthropomorphize robots more than tradi=onal domes=c technologies, meaning that an expecta=on of social intelligence may inherently result from a robot s design. This can be leveraged by portraying robots as being easy to communicate with, but can also lead to disappointed users when expecta=ons are not met Percep=on of Factors Affec=ng Acceptance Percep=on of Factors Affec=ng Acceptance! Percep=on of factors is as meaningful or ohen even more meaningful to domes=c users than the actual facts.! A. Previous experience: Being a primary source, these include personally- experienced life=me ac=ons and events as well as personally inferred beliefs, with educa=on and ini=al exposure being a large component of this factor. Previous experience with animals and children may be influen=al here as well. Given the new and unique nature of robots, a robot can be designed to influence users to draw on par=cular past experiences as desired by the designer.! B. Media: People's previous experience with robo=c technologies is very limited, so we expect them to strongly leverage media as an important source of informa=on. This includes classic science- fic=on- like literature, movies, and television, as well as more modern and fact- oriented news sources. Designing robots around media trends can be an important aspect of acceptance. 27! C. Personal social network: Opinions and perspec=ves offered by friends, neighbors and family have a large influence on how people perceive robots. Although robots are new and as such the social network itself will be less informed, this will likely be an important factor nonetheless. Although it is not clear how designers can influence this factor, somehow making a robot conducive to socializing would be helpful here. [?]! D. Robot design methodology: The design of a technology, its physical appearance, ac=ons, interface, and all other aspects of design, directly influence which previous experiences people use when forming their understanding of a given en=ty. With robots, designers can either leverage or constrain user tendencies to anthropomorphize. Designers may also use the robot's dynamic physical presence as means to influence percep=on, for example, by limi=ng speed or agility in an a;empt to convey a harmless or safe robot. Robots can use human social interac=on (gaze, facial expression, physical proximity) in new ways that other, previous technologies are unable to do. Moreover, eeriness phenomena may also be useful in the design of social robots, [e.g. ] to make domes=c users aware of poten=ally dangerous robots or situa=ons. 28 7

8 Evalua=ng long term human- robot interac=on: Leite et al Evalua=ng long term human- robot interac=on: Leite et al. 2013! a lot of work has been done in studying how users interact with robots within a single interac=on, only in the last decade the first long- term studies, in which the same user (or group of users) interacts with a robot several =mes, have started to appear.! Recently in Europe, several projects [on] long- term interac=ons with social robots and other agents (e.g., LIREC, Companions, SERA, CompanionAble and ALIZ- E).! Robots deployed in work environments and public spaces. e.g. Robotdalen s RobCab, a transporta=on robot for hospitals, the Siga Robots developed by YDreams to guide and interact with guests visi=ng the headquarters of Santander bank, and the robo=c characters developed by Walt Disney Imagineering for the Disney parks.! Four different applica=on domains: Health Care & Therapy, Educa=on, Work Environments & Public Spaces, and Home Health Care and Therapy Health Care and Therapy Table 1 Summary of the long-term studies in the health care and therapy domains References Agent/Robot Capabilities Exp. design Nr. sessions Main results Wada & Shibata (2006, 2007) Kidd & Breazeal (2008) Francois et al. (2009) Sabelli et al. (2011) Paro Autom AIBO Robovie Animal-like behaviour; responds to touch, sound and lights; limited-keyword recognition Eye contact and small talk depending on time of day, state of the relationship with the user, etc. Dog-like behaviour (e.g., wag the tail); responds to touch Remotely operated dialogues and child-like behaviours (e.g. what is this? ) Subjects: 12 Measures: degree of social interaction, stress levels Methods: video, interviews, urine tests Subjects: 45; years old (3 conditions) Measures: weight loss, WAI, usage of the system Methods: questionnaire Subjects: 6 (autistic children) Measures: children s progress during interaction Methods: video observation Subjects: 55 Measures: interaction patterns during interaction Methods: interviews, direct observations 30 (9 hours a day) Increased social interaction between participants, stress levels reduced 50 (average) Participants interacting with the robot reported their weight for more days and expressed more willing to continue interacting with the system 10 (40 minutes each) Children tended to express more interest towards the robot over time, with occasional displays of affect 15 to 35 (10 to 20 minutes each) Robot was well accepted due to role as child and behaviours such as greetings and calling users by their names

9 Educa=on Educa=on Table 2 Summary of the long-term studies in the domain of education References Robot Capabilities Exp. design Nr. sessions Main results Kanda et al. (2004) Robovie Identify users, recognising and speaking English Subjects: 228 Measures: length of interaction, English skills Methods: video observation, English tests 9 school days Interaction after 1th week declined; improvement of English skills in children who kept interacting with the robot Kanda et al. (2007) Robovie Identify users, pseudo-development mechanism, confiding personal information Subjects: 37 (10 11 years) Measures: length of interaction Methods: questionnaire, video observation 32 school days Children kept interacting with the robot after the 2nd week Salter et al. (2004) Wany Obstacle avoidance, move in the environment Subjects: 8 (5 8 years, male) Measures: activity around the robot Methods: video observation, analysis of interaction data 5 Childrenlostinterestin the interaction from the third session Tanaka et al. (2007) QRIO Choreographed dance sequences and mimicking children s movements Subjects: 11 (10 24 months) Measures: quality of interaction, haptic behaviour towards the robot Methods: video observation 15 (45 50 min. each) Toddlers progressively started treating QRIO as a peer and exhibited several care-taking behaviours towards the robot Educa=on Work Environments and Public Spaces Kozima et al. (2009) (study 1) Keepon Display non-verbal behaviours (gaze, emotions,...) Subjects: 27 (3 4 years) Measures: children s responses Methods: video observation 20 (90 minutes each) Robot played the role of social mediator; children maintained interest over the sessions Kozima et al. (2009) (study 2) Keepon Display non-verbal behaviours (gaze, emotions,...) Subjects: 30 (2 4 years, autistic) Measures: children s responses towards the robot Methods: video observation 15 Although eye contact decreased, children gradually approached the robot more and established physical contact Leite et al. (2008) icat Feedback on children s moves through facial expressions Subjects: 5 (5 15 years) Measures: social presence, eye contact with the robot Methods: questionnaire, video observation 5(aprox.1hour) Somedimensionsof social presence decreased; eye contact with the robot decreased after the 2nd week Hyun et al. (2010) irobiq Move head and arms, navigate in the environment, express emotions Subjects: 111 (5 years) Measures: children s perception of the robot Methods: interviews 10 (approx. 1 hour) Robots are well accepted by children in educational settings obots used in the long-term studies in work environments and public spaces (images used with permission of the authors)

10 Work Environments and Public Spaces Home Table 3 Summary of the long-term studies in work environments and public spaces References Robot Capabilities Exp. design Nr. sessions Main results Severinson- Eklundh et al. (2003) Cero Fetch-and-carry objects such as books or coffee cups Subjects: 1 target user in a work group of 30 Measures: long-term effects of a service robot Methods: video and direct observation, system logs, pos-trial interviews 66 Social robots in public spaces should be able to interact with everyone, not just the main users Stubbs et al. (2004) PER Simulated scientific testing Subjects: 11 Measures: people s cognitive model of the robot Methods: interviews 3 months Regular interactions influence people s cognitive model of the robot Gockley et al. (2005) Valerie Reveal back-story, recognise people around the booth, limited natural language user interaction through text input Subjects: 233 Measures: length of interactions Methods: analysis of interaction data 180 Many users kept interacting daily with the robot, but after a certain period only a few interacted for more than 30 seconds Kirby et al. (2007) Valerie Additional mood displays while telling stories Subjects: 62 Measures: length of interactions Methods: analysis of interaction data, questionnaire 45 (8 hours a day) Interaction patterns change according to the robot s mood and level of familiarity with the robot Kanda et al. (2010) Robovie Guiding, rapport building, identify repeated users, advertisement Subjects: 162 Measures: intention of use, interest, perceived familiarity, intelligence and adequacy of route guidance Methods: questionnaire 2.1 (average); from 2 to 18 sessions Perception of the robot was positive; shopping suggestions of the robot were accepted by visitors Home Leite et al. guidelines Table 4 Summary of the long-term studies in home environments References Robot Capabilities Exp. design Nr. sessions Main results Koay et al. (2003) Sung et al. (2009, 2010) Fernaeus et al. (2010) PeopleBot Approach the user in Subjects: 12 (8 male and several ways 4female) Measures: proxemic preferences Methods: questionnaire, comfort level device Roomba Vacuum cleaning, Subjects: 48 (across 30 move around the house households) Measures: acceptance of robot Methods: observation, interviews, probing techniques, activity cards, small questionnaires Pleo Animal-like behaviour Subjects: 6 families Measures: exploratory study Methods: interviews, video recordings and pictures 8 (aprox. 1 hour each) People s preferences in terms of promixity change over time 6 months Two months is the time required for observing stable interactions between robots and households. Several techniques should be complemented to really capture people s routines at home 2 10 months Initial expectations about Pleo were not met. After the novelty effect, participants played with the robot only occasionally Table 5 Summary of the guidelines for future design of social robots for long-term interaction Guideline Recommendations Appearance Select embodiment according to the robot s purpose and capabilities Functional embodiments well suited for home or office environments Animal-likeembodimentscreatelessexpectationsofrobot ssocialcapabilities Continuity and incremental behaviours Routinebehaviours(e.g.,greetingsandfarewells) Strategicbehaviours(e.g.,recallingpreviousactivitiesandself-disclosure) Incremental addition of novel behaviours over time Affective interactions and empathy Understand the user s affective state (and react accordingly) Displaycontextualisedaffectivereactions Memory and adaptation Identifynewandrepeatedusers Rememberaspectsofpastinteractionsandrecallthemappropriately Useinformationabouttheusertopersonalisetheinteraction Klamer et al. (2011) Nabaztag Personalised health conversations; users interact using yes- and no-buttons Subjects: 3 (50 65 years old, females) Measures: usage and acceptance of social robots Methods: interviews 10 days Utilitarian and social factors seem important reasons for participants to accept social robots in domestic environments

11 Lessons 1. Social robot = robot + social interface? 2. Care-o-bot: example of a *designed* robot 3. Programming is not the same as robot interaction design 4. Seven factors affect acceptance of domestic robots 5. Long term human-robot interaction studies can offer useful guidelines 41 11