NSF PI Workshop on Human-Robot Interaction

NSF PI Workshop on Human-Robot Interaction Survey Questions Question 1. To increase the use of robotic systems in real-world applications (such as home, health-care, hospitals, schools, search and rescue, etc.) would require concerted research efforts in advancing various interdisciplinary areas in human-robot interaction. In your opinion, which areas would achieve the most in advancing the field of HRI (please rank order their importance, 1 indicating most important): [ 2 ] learning, adaptation and SOCIAL LEARNING in HRI [ 3 ] user studies of HRI including ethnography and field studies and metrics for HRI evaluation [ ] design and evaluation of interfaces and/or haptics [ 1 ] modeling of HRI interactions (team-centric, task-centric, master-slave, peer-to-peer, etc.) including: HRI group dynamics and teamwork and: task allocation and coordination of HRI and: multi-modal perception, cognition and interaction and: system architectures for HRI and: long-term H-R relationships [ ] societal impact of HRI (risks, privacy, safety, ethics) [ ] awareness and monitoring of humans and their activities [ ] human behavior modeling Clearly all of these are important, so I m dubious to rank them explicitly. Especially given the Current breakdown. Some of these (i.e. perception and user modeling of humans, activity modeling, and interface design) are actively funded under different research communities, so I d be more interested in funding into HRI specific areas. Some of these easily falls under pattern recognition/machine perception/hci. All forms of human-robot evaluation studies are important. I won t rank them. This entails metrics, etc. But metrics cannot be developed out of context. I d include HRI modeling of interactions/hri teaming/task coordination/multi-modal percept/cog/act and HRI architecture to all be under the same problem domain. It s all about modeling intelligent interaction behaviors between robots and people. Question 2. For the three most important areas in the question above, what are the major components of those areas that you think are most important to address? Please comment and elaborate. Clearly my head is focused on interpersonal interactions between humans and robots. 1. Modeling of interaction skills in HRI. Synergistic partnership between humans and robots is key, whether one-on-one or many-to-one or one-to-many, etc. Attention to the compatibility of the robot s cognitive-social-affective behavior with human expectations 1

is critical. LONG-TERM interaction is very important and just starting to be explored. The acquisition of people-sense and common-sense in addition of skills and tasks continues to be a huge domain for robots to learn in order to be able to function effectively in the world and understand the world around them. 2. Learning and adaptation in HRI. I d include SOCIAL learning not just imitation. There are many forms of learning that robots are going to need in order to learn from anyone. Tutelage, demonstration, observation, emulation, etc. I d like to see a greater emphasis on robots learning in the wild where the natural environment includes people. The robots will have to start framing their own learning problems, and be socially guidable in the learning process without being completely dependent on a human being around in order to learn. Longitudinal learning. We need to start to think long and hard not just about the learning mechanisms, but the learning experience of these robots, and the role of the human in shaping that experience. If we re going to take learning seriously, then we need to take experience seriously --- and HRI could have a big impact in understanding this highly neglected topic. 3. User studies and evaluation in HRI. The evaluation/study will depend on the core scientific question. But I d like to see a push towards longitudinal studies over weeks, months, etc. Question 3. In your opinion, what are the emerging new technologies that would impact the most the field of HRI in the next 10 to 15 years, and how can HRI take advantage of these emerging technologies? As always, advances in computing and sensory systems. Leveraging ubiquitous computing environments as extended perception systems of robots Exploring the use of on-line games as a way to have thousands/millions of people interact with core algorithms, and think strongly about how to transfer these lessons/findings/data/experience to robots in the real-world with real people. How are we going to get a childhood of highquality learning experiences into robots? On-line games may be a valuable resource to draw from. But the transfer to real robots with real people is critical. Question 4. The field of HRI can have major impact on a wide range of real-world applications. In your opinion, what are the application domain(s) that would benefit the most from advances of HRI? Please rank order 3 most important applications and give a brief justification for your selection. [ 1 ] assistive robotics (e.g., rehab, assistive living, eldercare, aging in place, etc.) and health-care applications (e.g., hospitals, nursing homes, medical applications, etc.) [ ] entertainment [ 3 ] homeland security and search and rescue operation 2

[ ] service robotics (e.g., tour guide in museums, schools, & department stores; companionship; service in fast-food stores, etc.) [ ] education [ 2 ] supplementing human workforces (e.g., human-robot team work) [ ] domesticated robotics (e.g., intelligent homes) Comments: Everyone knows of the aging population issue. That will continue to drive many applications in robotics. Robots in the homes helping people to live independently longer. I see assistive robotics and heal-care to be the same category. Related to this, with a diminishing work force, supplementing human labor with robot labor will also matter. Rod Brooks has a whole story to tell here --- robots working alongside people. Agriculture, assembly, etc. Probably homeland security and search and rescue after that. Helping people while in need helping to keep human rescuers out of harm s way as much as possible. I see intelligent homes as being a different discipline than HRI. Question 5. What is your expectation from attending and participating in this NSF Workshop on HRI? Important scientific discussion for how to best advance progress and funding support in this field. Identifying critical topics and ideas not on this survey. Question 6. Briefly state any questions or comments that you may have for the Workshop panelists and organizers that may be of broad interest to the participants. Currently, the NIH does not have many programs to support the development of new (robotic) technologies in the area of health and medicine, although they do support studies. This presents a significant hole in funding opportunities for HRI. The NSF is one of the few organizations that will fund the development of new technological systems. I d like to see this increase given do societal demands in the next few decades. HRI work is often multi-disciplinary and requires larger teams than one professor and a student to tackle a problem thoroughly. I d like to see the NSF support the funding larger, multidisciplinary teams in HRI. There has been an on-going discussion of a journal dedicated to HRI. If there s time, we might want to discuss if this is the right time to establish such a journal (i.e., sufficient critical mass, maturity of the discipline, etc) 3

NSF PI Workshop on Human-Robot Interaction Survey Questions Question 1. To increase the use of robotic systems in real-world applications (such as home, health-care, hospitals, schools, search and rescue, etc.) would require concerted research efforts in advancing various interdisciplinary areas in human-robot interaction. In your opinion, which areas would achieve the most in advancing the field of HRI (please rank order their importance, 1 indicating most important): [6] learning, adaptation and imitation in HRI [7] user studies of HRI [8] ethnography and field studies [5] multi-modal perception, cognition and interaction [4] design and evaluation of interfaces and/or haptics [13] system architectures for HRI [11] metrics for HRI evaluation [3] human behavior modeling [10] modeling of HRI interactions (team-centric, task-centric, master-slave, peer-to-peer, etc.) [9] HRI group dynamics and teamwork [12] task allocation and coordination of HRI [1] societal impact of HRI (risks, privacy, safety, ethics) [2] awareness and monitoring of humans and their activities Question 2. For the three most important areas in the question above, what are the major components of those areas that you think are most important to address? Please comment and elaborate. [1] societal impact of HRI (risks, privacy, safety, ethics) [2] awareness and monitoring of humans and their activities [3] human behavior modeling In any effective relationship, those engaged need to be committed to the partnership. There is no reason to believe this will not be true in human-robot relationships: the H in HRI will need to be fully on board for effective interaction to occur. In each of the above three areas, the most important components are those that promote human engagement in human-robot relationships by serving 1) to ease apprehension often suffered by those interacting with robots, 2) to create an interaction experience that, through familiarity, avoids cognitive overload, and 3) to sustain a sensitivity to the ethical dimensions of this interaction, a sensitivity expected as a matter of course by human beings in their relationships. Question 3. In your opinion, what are the emerging new technologies that would impact the most the field of HRI in the next 10 to 15 years, and how can HRI take advantage of these emerging technologies? Machine Ethics 1

Past research concerning the relationship between technology and ethics has largely focused on responsible and irresponsible use of technology by human beings, with a few people being interested in how human beings ought to treat machines. In all cases, only human beings have engaged in ethical reasoning. The time has come for adding an ethical dimension to at least some machines. Recognition of the ethical ramifications of recent and potential developments in machine autonomy necessitates this. In contrast to computer hacking, software property issues, privacy issues and other topics normally ascribed to computer ethics, machine ethics is concerned with the behavior of machines towards human users and other machines. The goal of machine ethics is to create a machine that follows an acceptable ethical principle or set of principles, that is to say, it is guided by this principle or these principles in the decisions it makes about possible courses of action it could take. The behavior of more fully autonomous machines, guided by such an ethical dimension, is likely to be more acceptable in real-world environments than that of machines without such a dimension. We contend that research in machine ethics is key to alleviating concerns with autonomous systems it could be argued that the notion of such machines without an ethical dimension is at the root of all fears concerning machine intelligence. This nascent research area has seen much recent activity including a AAAI symposium and special sessions in ALifeX, the North American Computers and Philosophy Conference, and the upcoming Eastern Division Meeting of the American Philosophical Association. Further, a special issue of IEEE Intelligent Systems on Machine Ethics has just been published and two books on the subject (one by J. Storrs Hall and the other by Colin Allen and Wendell Wallace) are slated to appear soon. Importantly, incorporating ethical principles in HRI systems will not only constrain them from doing harm but 1) could permit them to engage in tasks in which they might not otherwise be permitted to engage without such an ethical dimension and 2) will not only permit them to complete such tasks in a more ethically sensitive manner but will also promote more effective and efficient means to do so. For example, reminder systems guided by a principle concerning the relationships between the duties of nonmaleficence, beneficence, and respect for autonomy can be more effective by keying the frequency and timing of reminders and notifications to overseers to these duties, balancing the relative importance of the duties involved maximizing good while minimizing harm and the frequency of these reminders and notifications. This ethical principle can, further, serve as an efficient abstraction that shields the developer from the complexity of monitoring a wide variety and number of such reminders in an ethically sensitive manner. This example shows that systems do not require much autonomy or intelligence to be faced with an ethical dilemma. Question 4. The field of HRI can have major impact on a wide range of real-world applications. In your opinion, what are the application domain(s) that would benefit the most from advances of HRI? Please rank order 3 most important applications and give a brief justification for your selection. [2] assistive robotics (e.g., rehab, assistive living, etc.) [1] health-care applications (e.g., hospitals, nursing homes, medical applications, etc.) 2

[ ] entertainment [ ] homeland security [ ] search and rescue operation [ ] service robotics (e.g., tour guide in museums, schools, & department stores; companionship; service in fast-food stores, etc.) [ ] education [ ] manufacturing and assembly (e.g., human-robot team work) [3] domesticated robotics (e.g., intelligent homes) Comments: [1] health-care applications (e.g., hospitals, nursing homes, medical applications, etc.) [2] assistive robotics (e.g., rehab, assistive living, etc.) [3] domesticated robotics (e.g., intelligent homes) Given the quickly aging population, HRI applications in areas of health care, rehab, and assisted living are likely to provide welcomed relief to an overwhelmed health care system. Domesticated robotics is also likely to help this effort indirectly by permitting the aged to remain in their homes longer than they might otherwise be able without the household help provided by domestic systems. Question 5. What is your expectation from attending and participating in this NSF Workshop on HRI? It is my hope that the HRI community gains an understanding of the feasibility and necessity of machine ethics and the importance of such research to the endeavor of HRI. As full autonomy and intelligence are not necessary for ethical considerations to arise, I am also interested in uncovering ethical dilemmas already faced by HRI systems and determining how machine ethics could be useful in resolving them. Question 6. Briefly state any questions or comments that you may have for the Workshop panelists and organizers that may be of broad interest to the participants. What ethical dilemmas are faced by current HRI systems and how are they dealt with? Further, what will be those faced by future HRI systems and what are the plans for dealing with them? 3

NSF PI Workshop on Human-Robot Interaction Survey Questions Question 1. To increase the use of robotic systems in real-world applications (such as home, health-care, hospitals, schools, search and rescue, etc.) would require concerted research efforts in advancing various interdisciplinary areas in human-robot interaction. In your opinion, which areas would achieve the most in advancing the field of HRI (please rank order their importance, 1 indicating most important): [7] learning, adaptation and imitation in HRI [12] user studies of HRI [13] ethnography and field studies [3] multi-modal perception, cognition and interaction [6] design and evaluation of interfaces and/or haptics [11] system architectures for HRI [10] metrics for HRI evaluation [5] human behavior modeling [8] modeling of HRI interactions (team-centric, task-centric, master-slave, peer-to-peer, etc.) [9] HRI group dynamics and teamwork [4] task allocation and coordination of HRI [2] societal impact of HRI (risks, privacy, safety, ethics) [1] awareness and monitoring of humans and their activities Question 2. For the three most important areas in the question above, what are the major components of those areas that you think are most important to address? Please comment and elaborate. Issues 1-5 (and partially of 6 and 7) are important because safe and unburdened interaction with robotics is the most significant problem. To address this underlying fundamental issue, research must address the question of what is the human s intent and how can the robot execute that intent. For interactions where the robot is corrective to the human (e.g., rehabilitation therapy) the question what is the human s intent may be difficult to answer because the robot will have to distinguish between the desired action and a disturbance action (e.g., muscle spasticity) that needs to be rejected. Answers to these questions involve awareness of human action (issue 1,2,3,5,7) and then task allocation to determine what part of the task is the robot s responsibility or the human s (issue 4, 6, 8, 9). Question 3. In your opinion, what are the emerging new technologies that would impact the most the field of HRI in the next 10 to 15 years, and how can HRI take advantage of these emerging technologies? Tissue interfaces such as interfaces with the brain, with muscles, nerve endings, and organs. Tissues interfaces provide a basic input and output element for HRI. These are the basic hooks that will allow technologies to be integrated. 1

Question 4. The field of HRI can have major impact on a wide range of real-world applications. In your opinion, what are the application domain(s) that would benefit the most from advances of HRI? Please rank order 3 most important applications and give a brief justification for your selection. [1] assistive robotics (e.g., rehab, assistive living, etc.) [2] health-care applications (e.g., hospitals, nursing homes, medical applications, etc.) [ ] entertainment [ ] homeland security [ ] search and rescue operation [3] service robotics (e.g., tour guide in museums, schools, & department stores; companionship; service in fast-food stores, etc.) [ ] education [ ] manufacturing and assembly (e.g., human-robot team work) [ ] domesticated robotics (e.g., intelligent homes) Comments: Application 1 and 2 can exploit robotic technology to make the most dramatic changes in quality of life and activities of daily living. Application 3 is the most pervasive application that fits in exactly with a service and information oriented economy. Question 5. What is your expectation from attending and participating in this NSF Workshop on HRI? Gain an understanding of the issues researchers are addressing. Provide insight regarding the technologies I am developing. Search for inspiration for new technology development. Question 6. Briefly state any questions or comments that you may have for the Workshop panelists and organizers that may be of broad interest to the participants. 2

NSF PI Workshop on Human-Robot Interaction Survey Questions Question 1. To increase the use of robotic systems in real-world applications (such as home, health-care, hospitals, schools, search and rescue, etc.) would require concerted research efforts in advancing various interdisciplinary areas in human-robot interaction. In your opinion, which areas would achieve the most in advancing the field of HRI (please rank order their importance, 1 indicating most important): [ 4 ] learning, adaptation and imitation in HRI [ 2 ] user studies of HRI [ 2 ] ethnography and field studies [ 4 ] multi-modal perception, cognition and interaction [ 4 ] design and evaluation of interfaces and/or haptics [ 1 ] system architectures for HRI [ 1 ] metrics for HRI evaluation [ 5 ] human behavior modeling [ 6 ] modeling of HRI interactions (team-centric, task-centric, master-slave, peer-to-peer, etc.) [ 2 ] HRI group dynamics and teamwork [ 1 ] task allocation and coordination of HRI [ 7 ] societal impact of HRI (risks, privacy, safety, ethics) [ 3 ] awareness and monitoring of humans and their activities Question 2. For the three most important areas in the question above, what are the major components of those areas that you think are most important to address? Please comment and elaborate. System architectures for HRI: We need software frameworks that facilitate the creation and operation of joint human-robot teams and/or make it easier for humans and robots to communicate/coordinate their actions. Conventional robot control architectures do not adequately address HRI needs, particularly if human and robot engage in high-level dialogue. We also need architectures that allow different researchers and groups to share and reuse code. Metrics for HRI evaluation: We need to establish common metrics and evaluation procedures so that researchers can discuss and assess each other s work. Well-defined and comprehensible metrics are also essential for system designers (in NASA, DoD, etc.) to understand the impact of employing a human-robot team. Steinfeld, A., Fong, T., et al. Common metrics for human-robot interaction (HRI2006) is a good start, but it s only a start. Task allocation and coordination of HRI: We need to better understand when it is possible to allocate tasks between humans and robots, and how best to do this. We also need to understand when it is better for human and robot to work jointly on a task. Coordination clearly involves human activity monitoring, and perhaps human intent prediction / human modeling. 1

Question 3. In your opinion, what are the emerging new technologies that would impact the most the field of HRI in the next 10 to 15 years, and how can HRI take advantage of these emerging technologies? Real-time simulation (particularly physics and model based) could be used for rapid prototyping and evaluating HRI. Computational cognitive architectures could be used to model human behavior and make human and robot more understandable to each other, so that interaction becomes more humancompatible. Perceptual user interfaces (particularly vision-based) could provide additional interaction modalities and thus make HRI richer and more fluid. Question 4. The field of HRI can have major impact on a wide range of real-world applications. In your opinion, what are the application domain(s) that would benefit the most from advances of HRI? Please rank order 3 most important applications and give a brief justification for your selection. [ ] assistive robotics (e.g., rehab, assistive living, etc.) [ ] health-care applications (e.g., hospitals, nursing homes, medical applications, etc.) [ ] entertainment [x ] homeland security [x ] search and rescue operation [ ] service robotics (e.g., tour guide in museums, schools, & department stores; companionship; service in fast-food stores, etc.) [ ] education [ ] manufacturing and assembly (e.g., human-robot team work) [ ] domesticated robotics (e.g., intelligent homes) [ x ] other: space_exploration Comments: I believe that HRI has the potential to significantly improve task-oriented human-robot teams, particularly for NASA, DoD, and DHS missions. In these domains, the more the human-robot team is effective and productive, the more successful the mission will be (i.e., in terms of reduced risk, reduced, cost, etc). For example, if NASA is able to use robots to reduce the number of extra-vehicular activity (EVA) sorties that astronauts have to perform, or the duration of those sorties, or both, then that will fundamentally change the way space exploration is performed. Regarding the other application areas, it is extremely important to recognize that societal and cultural differences play a very significant role in the acceptance of robots in those areas. For example, the potential for health-care or assistive applications to benefit from HRI, is closely tied to whether or not a particular society/culture is willing to accept robotics in health-care or assistance. There are vast differences between asian (especially Japan) markets and the American market. Question 5. What is your expectation from attending and participating in this NSF Workshop on HRI? 2

The stated goal of the workshop is gathering a diverse group of researchers to explore and discuss all the important and emerging areas in human-robot interaction (HRI) that will have major impact on this cross-cutting focus area. Although the invited group is fairly diverse, it could be improved: (1) only the American research community was represented (why were there no Europeans, Japanese, or Koreans?); (2) several prominent American HRI researchers were not invited (Sara Kiesler, Bill Clancey, Pam Hinds, Robin Murphy, Reid Simmons, Julie Adams, Jeff Bradshaw, Jean Scholtz, Aaron Steinfeld), (3) several people were invited who are not HRI researchers (Ralph Hollis, Warren Dixon, Gunter Niemeyer, Manuela Veloso), and (4) more representation from other agencies (NASA, ONR, DARPA, NIST) would have been very helpful. Question 6. Briefly state any questions or comments that you may have for the Workshop panelists and organizers that may be of broad interest to the participants. What is the definition of HRI? Is it a discipline, a noun, a verb, all of these? How can we best setup and maintain a global online HRI community presence? What resources are most needed: Email listserv? Wiki? Paper archive? Shared testbed (simulation or multiple robots)? Several groups of researchers have recently been discussing starting a HRI journal. What recommendations do people have? How can we improve the two primary HRI conferences: HRI and RO-MAN? What distinguishes HRI from HCI, human factors, design, etc.? For example, do we consider manual control (direct teleoperation) to be HRI? What parts of HRI do we know enough about (or can see that we will sometime soon know enough about) to provide methodology, guidelines, and standards? What role can and should standards (for design, integration, evaluation) play in HRI? Can we design standardized interactions or interfaces for specific applications (or classes of applications)? 3

NSF PI Workshop on Human-Robot Interaction Question 1. To increase the use of robotic systems in real-world applications (such as home, health-care, hospitals, schools, search and rescue, etc.) would require concerted research efforts in advancing various interdisciplinary areas in human-robot interaction. In your opinion, which areas would achieve the most in advancing the field of HRI (please rank order their importance, 1 indicating most important): [5] learning, adaptation and imitation in HRI [4] user studies of HRI [ ] ethnography and field studies [2] multi-modal perception, cognition and interaction [1] design and evaluation of interfaces and/or haptics [ ] system architectures for HRI [ ] metrics for HRI evaluation [ ] human behavior modeling [ ] modeling of HRI interactions (team-centric, task-centric, master-slave, peer-to-peer, etc.) [ ] HRI group dynamics and teamwork [ ] task allocation and coordination of HRI [ ] societal impact of HRI (risks, privacy, safety, ethics) [3] awareness and monitoring of humans and their activities Comments: All of these are important. I am assuming that user studies would be included in my top 3 picks, so I did not rank it higher. Question 2. For the three most important areas in the question above, what are the major components of those areas that you think are most important to address? Please comment and elaborate. 1. design and evaluation of interfaces and/or haptics In the short term, much can be gained by coupling the current state of the art in robot autonomy with interfaces designed to take advantage of the user s advanced perception and reasoning skills. Interfaces are also critical for medical and rehabilitation applications. Components to address include: Understanding typical human to human interfaces that can be leveraged (in many modalities, including multiple modalities) Understanding the human sensorimotor system Understanding human cognition used in interfaces For specific tasks, understanding the human task with enough fidelity to identify potential areas in which a robot + human can perform better than the human alone Methods to evaluate the interfaces, including how to compare the performance of different interfaces 2. multi-modal perception, cognition and interaction This covers a wide range. It seems that perception and cognition are always challenges. Components to address include: Understanding and processing visual sensory data fast enough to be useful in real time 1

More than just low level processing of visual data, we need high level understanding of visual (potentially changing) scenes. Given an abundance of multi-modal sensory data, what data is most important for the current task or mission How can human models of cognition be leveraged for providing better reasoning and for better (more natural) human-robot interaction 3. awareness and monitoring of humans and their activities If robots are to work with humans in an assistive or collaborative mode, they must be able to sense and determine what the human is doing or when the human is having difficulty. Components to address include: Reliable and robust recognition, identification and tracking of human partners in unstructured environments Reliable recognition of a human pose in unstructured environments Recognition of emotional stress in human partners/users The ability to segment human activities, e.g., interpret pauses for dialogue and taking turns The ability to recognize task segments, reliably, in unstructured environments Question 3. In your opinion, what are the emerging new technologies that would impact the most the field of HRI in the next 10 to 15 years, and how can HRI take advantage of these emerging technologies? New information in neuroscience can provide inspiration for creating adaptive systems and for understanding human collaborators. New information on the human sensorimotor system can provide new methods and interfaces for assistive and rehabilitation robots. Question 4. The field of HRI can have major impact on a wide range of real-world applications. In your opinion, what are the application domain(s) that would benefit the most from advances of HRI? Please rank order 3 most important applications and give a brief justification for your selection. [2] assistive robotics (e.g., rehab, assistive living, etc.) [1] health-care applications (e.g., hospitals, nursing homes, medical applications, etc.) [ ] entertainment [ ] homeland security [3] search and rescue operation [ ] service robotics (e.g., tour guide in museums, schools, & department stores; companionship; service in fast-food stores, etc.) [ ] education [ ] manufacturing and assembly (e.g., human-robot team work) [ ] domesticated robotics (e.g., intelligent homes) Comments: It would be easier for HRI researchers to work on applications that do not have such serious consequences in case of failures, such as entertainment and service robots (e.g., museum tour 2

guides). However, the application domains that would stand to really benefit from HRI work include more serious applications. Robots have already shown their value in the medical applications, and in the other selected application areas, there is good potential. Question 5. What is your expectation from attending and participating in this NSF Workshop on HRI? I would like to broaden my horizons by meeting more HRI researchers and in areas where I have not worked. This may provide good potential for new ideas or using old methods for new purposes. I also want to find out what other HRI researchers consider to be the important areas to address and the range of problems being explored. Question 6. Briefly state any questions or comments that you may have for the Workshop panelists and organizers that may be of broad interest to the participants. 3

NSF PI Workshop on Human-Robot Interaction Survey Questions Question 1. To increase the use of robotic systems in real-world applications (such as home, health-care, hospitals, schools, search and rescue, etc.) would require concerted research efforts in advancing various interdisciplinary areas in human-robot interaction. In your opinion, which areas would achieve the most in advancing the field of HRI (please rank order their importance, 1 indicating most important): [2] learning, adaptation and imitation in HRI [10] user studies of HRI [11] ethnography and field studies [1] multi-modal perception, cognition and interaction [3] design and evaluation of interfaces and/or haptics [4] system architectures for HRI [8] metrics for HRI evaluation [5] human behavior modeling [6] modeling of HRI interactions (team-centric, task-centric, master-slave, peer-to-peer, etc.) [9] HRI group dynamics and teamwork [7] task allocation and coordination of HRI [9] societal impact of HRI (risks, privacy, safety, ethics) [5] awareness and monitoring of humans and their activities Question 2. For the three most important areas in the question above, what are the major components of those areas that you think are most important to address? Please comment and elaborate. The components are equally important and will vary depending on the applications considered. The below view is more perception-centric, but actuation issues are equally important and should not be considered separately. Topics ranked as [1] focuses on incorporating multi-modal sensory information (visual, tactile, speech/audio and various special purpose sensors) and developing computational models for integrating it. The integration should be driven by the tasks and/or goals to be achieved. For example development of computer vision algorithms visual for detection and recognition of objects and/or human activities can be advance human-robot interactions in home, health-care and rescue environments and in the context of various activity monitoring and detection/search tasks. The issues of designing better interfaces rests on robots being able better understand human activities as well as environments where they co-exist, hence topic [3]. Issues of incremental learning and adaptation would play an important role in this context. Advancement in tactile sensing and haptics interfaces are essential for certain applications [2] in order to facilitate better manipulation, sensing capabilities. 1

Question 3. In your opinion, what are the emerging new technologies that would impact the most the field of HRI in the next 10 to 15 years, and how can HRI take advantage of these emerging technologies? Question 4. The field of HRI can have major impact on a wide range of real-world applications. In your opinion, what are the application domain(s) that would benefit the most from advances of HRI? Please rank order 3 most important applications and give a brief justification for your selection. [1] assistive robotics (e.g., rehab, assistive living, etc.) [2] health-care applications (e.g., hospitals, nursing homes, medical applications, etc.) [6] entertainment [5] homeland security [4] search and rescue operation [3] service robotics (e.g., tour guide in museums, schools, & department stores; companionship; service in fast-food stores, etc.) [3] education [ ] manufacturing and assembly (e.g., human-robot team work) [ ] domesticated robotics (e.g., intelligent homes) Comments: Question 5. What is your expectation from attending and participating in this NSF Workshop on HRI? I would like to become more familiar with existing application domains and tasks and explore they can benefit from most recent advances in computer vision, multi-modal sensing/interfaces, task modeling, learning and control. Question 6. Briefly state any questions or comments that you may have for the Workshop panelists and organizers that may be of broad interest to the participants. Following similar trends in the robotics and computer vision it would be useful to identify/discuss some challenges which would be specific to HRI (e.g. DARPA grand challenge, AAAI robot competitions). I realize that the variety of issues to be dealt with in HRI is much broader and multi-disciplinary 2

than the examples above, but it may be useful to speculate along these lines. It might be also useful to explore possibilities and avenues for researchers for sharing datasets and/or some of the infrastructure, which would enable broader participation of the community 3

NSF PI Workshop on Human-Robot Interaction Survey Questions Question 1. To increase the use of robotic systems in real-world applications (such as home, health-care, hospitals, schools, search and rescue, etc.) would require concerted research efforts in advancing various interdisciplinary areas in human-robot interaction. In your opinion, which areas would achieve the most in advancing the field of HRI (please rank order their importance, 1 indicating most important): [ 1 ] learning, adaptation and imitation in HRI [ 7 ] user studies of HRI [ 11 ] ethnography and field studies [ 2 ] multi-modal perception, cognition and interaction [ 6 ] design and evaluation of interfaces and/or haptics [ 13 ] system architectures for HRI [ 8 ] metrics for HRI evaluation [ 5 ] human behavior modeling [ 10 ] modeling of HRI interactions (team-centric, task-centric, master-slave, peer-to-peer, etc.) [ 12 ] HRI group dynamics and teamwork [ 4 ] task allocation and coordination of HRI [ 9 ] societal impact of HRI (risks, privacy, safety, ethics) [ 3 ] awareness and monitoring of humans and their activities Question 2. For the three most important areas in the question above, what are the major components of those areas that you think are most important to address? Please comment and elaborate. (1) When to learn, what to learn, how to self-configure the representation of what is learned, how to autonomously derive taxonomies from raw data, what to forget, when to forget? (2) What intermediary representation is required which lies between the raw sensor space (e.g., array of pixels from a camera) and the cognitive space (e.g., visual situation awareness)? (3) What are the goals and intentions of humans and how can they be assisted by the HRI system. Question 3. In your opinion, what are the emerging new technologies that would impact the most the field of HRI in the next 10 to 15 years, and how can HRI take advantage of these emerging technologies? Recent advances in biologically-inspired algorithms and techniques are beginning to show a new breed of systems with unparalleled online trainability, adaptability, robustness to environmental variations, and low power requirements. Such neuromorphic engineering systems will be crucial to the development of effective and compliant HRI. 1

Question 4. The field of HRI can have major impact on a wide range of real-world applications. In your opinion, what are the application domain(s) that would benefit the most from advances of HRI? Please rank order 3 most important applications and give a brief justification for your selection. [ ] assistive robotics (e.g., rehab, assistive living, etc.) [ 3 ] health-care applications (e.g., hospitals, nursing homes, medical applications, etc.) [ ] entertainment [ ] homeland security [ ] search and rescue operation [ 1 ] service robotics (e.g., tour guide in museums, schools, & department stores; companionship; service in fast-food stores, etc.) [ ] education [ ] manufacturing and assembly (e.g., human-robot team work) [ 2 ] domesticated robotics (e.g., intelligent homes) Comments: The chosen domains are within reasonable reach over a 5-10 year time span, and I expect that HRI could impact these domains significantly in the near future, especially through the development of neuromorphic strategies for sensing, reasoning and acting. Question 5. What is your expectation from attending and participating in this NSF Workshop on HRI? To learn more about HRI, meet key players in the field. Question 6. Briefly state any questions or comments that you may have for the Workshop panelists and organizers that may be of broad interest to the participants. Thanks for organizing this! 2

NSF PI Workshop on Human-Robot Interaction Survey Questions Question 1. To increase the use of robotic systems in real-world applications (such as home, health-care, hospitals, schools, search and rescue, etc.) would require concerted research efforts in advancing various interdisciplinary areas in human-robot interaction. In your opinion, which areas would achieve the most in advancing the field of HRI (please rank order their importance, 1 indicating most important): [ 3 ] learning, adaptation and imitation in HRI [ 9 ] user studies of HRI [ 10] ethnography and field studies [ 11] multi-modal perception, cognition and interaction [ 5 ] design and evaluation of interfaces and/or haptics [ 12] system architectures for HRI [ 7 ] metrics for HRI evaluation [ 4 ] human behavior modeling [ 1 ] modeling of HRI interactions (team-centric, task-centric, master-slave, peer-to-peer, etc.) [ 2 ] HRI group dynamics and teamwork [ 6 ] task allocation and coordination of HRI [ 13] societal impact of HRI (risks, privacy, safety, ethics) [ 8 ] awareness and monitoring of humans and their activities Question 2. For the three most important areas in the question above, what are the major components of those areas that you think are most important to address? Please comment and elaborate. 1. Modeling HRI interactions a. Understanding roles and capabilities of humans and robots b. Creating predictive models that support evaluation. c. Understanding dynamic autonomy, mixed initiative, and information exchange 2. HRI group dynamics a. Number of humans to robots ratio b. Hand-off of task management between humans c. Social pressures, and social cues d. Role management 3. Learning a. Humans adapting to robot capabilities b. Robots learning from and being trained by humans c. Training of humans to use robots d. Managing and framing expectations Question 3. In your opinion, what are the emerging new technologies that would impact the most the field of HRI in the next 10 to 15 years, and how can HRI take advantage of these emerging technologies? 1. Natural language understanding: supporting interactions via something other than a GUI 1

2. Augmented reality: improving information exchange by mixing sensor, semantic, and intentional information 3. Autonomy and sensors probabilistic robotics: increased neglect tolerance and improved capacity to perform meaningful behaviors Question 4. The field of HRI can have major impact on a wide range of real-world applications. In your opinion, what are the application domain(s) that would benefit the most from advances of HRI? Please rank order 3 most important applications and give a brief justification for your selection. [ 1 ] assistive robotics (e.g., rehab, assistive living, etc.) [ 4 ] health-care applications (e.g., hospitals, nursing homes, medical applications, etc.) [ ] entertainment [ 2 ] homeland security [ 3 ] search and rescue operation [ ] service robotics (e.g., tour guide in museums, schools, & department stores; companionship; service in fast-food stores, etc.) [ 6 ] education [ ] manufacturing and assembly (e.g., human-robot team work) [ ] domesticated robotics (e.g., intelligent homes) [ 5 ] other: pollution monitoring and mitigation, disaster response, toxic spill management Comments: The rankings are a bit arbitrary, but my general priority list is (a) directly helping those who are in need, (b) preventing and mitigating natural or man-made threats, and (c) accomplishing something that is not easily done by humans and which robots improve performance. Question 5. What is your expectation from attending and participating in this NSF Workshop on HRI? Expand my vision for what can be accomplished in HRI, improve my understanding of what is being done, and form strategies for how to span the difference. Question 6. Briefly state any questions or comments that you may have for the Workshop panelists and organizers that may be of broad interest to the participants. There are efforts to create a wiki and webpages with HRI-related information. The temporary venues include this workshop s homepage, the HRI2007 Related Workshops page, and the HRI Journal wiki hrijournal.pbwiki.com. 2

NSF PI Workshop on Human-Robot Interaction Survey Questions Question 1. To increase the use of robotic systems in real-world applications (such as home, health-care, hospitals, schools, search and rescue, etc.) would require concerted research efforts in advancing various interdisciplinary areas in human-robot interaction. In your opinion, which areas would achieve the most in advancing the field of HRI (please rank order their importance, 1 indicating most important): [5] learning, adaptation and imitation in HRI [7] user studies of HRI [6] ethnography and field studies [1] multi-modal perception, cognition and interaction [9] design and evaluation of interfaces and/or haptics [10] system architectures for HRI [8] metrics for HRI evaluation [2] human behavior modeling [12] modeling of HRI interactions (team-centric, task-centric, master-slave, peer-to-peer, etc.) [13] HRI group dynamics and teamwork [11] task allocation and coordination of HRI [3] societal impact of HRI (risks, privacy, safety, ethics) [4] awareness and monitoring of humans and their activities Question 2. For the three most important areas in the question above, what are the major components of those areas that you think are most important to address? Please comment and elaborate. In my opinion, the most fundamental missing piece from most HRI systems is a reasonable representation of a human's goals, intentions, and activities. While our understanding of how to build complex systems has continued to advance greatly, we still spend most of our design time trying to find ways to "cheat" the perceptual problem. Our robots need to understand not only what activities that a human is performing, but also to be able to intuit the purpose behind those actions. Question 3. In your opinion, what are the emerging new technologies that would impact the most the field of HRI in the next 10 to 15 years, and how can HRI take advantage of these emerging technologies? Our ability to deliver safe, reliable systems at low cost to the home user is going to fundamentally change the types of studies that we are capable of performing. 1

Question 4. The field of HRI can have major impact on a wide range of real-world applications. In your opinion, what are the application domain(s) that would benefit the most from advances of HRI? Please rank order 3 most important applications and give a brief justification for your selection. [1] assistive robotics (e.g., rehab, assistive living, etc.) [2] health-care applications (e.g., hospitals, nursing homes, medical applications, etc.) [ ] entertainment [ ] homeland security [3] search and rescue operation [ ] service robotics (e.g., tour guide in museums, schools, & department stores; companionship; service in fast-food stores, etc.) [ ] education [ ] manufacturing and assembly (e.g., human-robot team work) [ ] domesticated robotics (e.g., intelligent homes) Comments: Most of the other domains in the foreseeable future will continue to be dominated by human workers. I chose three domains where interactive robots can perform tasks that humans cannot. Question 5. What is your expectation from attending and participating in this NSF Workshop on HRI? I'm hoping to get some insight from the funding agencies on their priorities for HRI work in the near future. Question 6. Briefly state any questions or comments that you may have for the Workshop panelists and organizers that may be of broad interest to the participants. I'd like to see some discussion on how to foster interdisciplinary work within the context of HRI. 2

NSF PI Workshop on Human-Robot Interaction Survey Questions Question 1. To increase the use of robotic systems in real-world applications (such as home, health-care, hospitals, schools, search and rescue, etc.) would require concerted research efforts in advancing various interdisciplinary areas in human-robot interaction. In your opinion, which areas would achieve the most in advancing the field of HRI (please rank order their importance, 1 indicating most important): [ 6 ] learning, adaptation and imitation in HRI [ ] user studies of HRI [ ] ethnography and field studies [ 2 ] multi-modal perception, cognition and interaction [ ] design and evaluation of interfaces and/or haptics [ ] system architectures for HRI [ ] metrics for HRI evaluation [ 1 ] human behavior modeling [ 3 ] modeling of HRI interactions (team-centric, task-centric, master-slave, peer-to-peer, etc.) [ 5 ] HRI group dynamics and teamwork [ ] task allocation and coordination of HRI [ ] societal impact of HRI (risks, privacy, safety, ethics) [ 4 ] awareness and monitoring of humans and their activities Question 2. For the three most important areas in the question above, what are the major components of those areas that you think are most important to address? Please comment and elaborate. Actually, I believe that to really solve HRI you need to look at integration across multiple disciplines so its hard to say that one of the above areas is most important. The devil is really in the integration using multiple techniques and modalities to disambiguate meaning, etc. I do believe we need to have our robots accommodate humans, and to that end, studying and modeling human behavior, particular when they collaborate and work in teams, and then building computational models of the human-to-human behavior and using those models as reasoning mechanisms in the robots will enable much better, natural interactions. Secondly, the notion of common ground between the human and robot is important. To help disambiguate meaning, humans use a lot of context about the shared world. The robots need to have this common ground. Question 3. In your opinion, what are the emerging new technologies that would impact the most the field of HRI in the next 10 to 15 years, and how can HRI take advantage of these emerging technologies? Better machine perception is important, and advances in flash ladar will help with giving the robot the ability perceive the environment important for achieving common ground. Advances in cognitive neuroscience will give us better understanding of how we process senses and better understanding of how we represent information. 1