Autism and Nao: It s a Very Compelling Device

Size: px

Start display at page:

Download "Autism and Nao: It s a Very Compelling Device"

Rudolf Scott
6 years ago
Views:

1 Autism and Nao: It s a Very Compelling Device Re-published with permission from Family Center on Technology and Disability Technology Voices October 2010

Autism and Nao: It s a Very Compelling Device An Interview with Notre Dame Nao senior team members Dr. Joshua Diehl, Assistant Professor; Dr. Charles Crowell, Associate Professor and Dr.

2 Autism and Nao: It s a Very Compelling Device An Interview with Notre Dame Nao senior team members Dr. Joshua Diehl, Assistant Professor; Dr. Charles Crowell, Associate Professor and Dr. Michael Villano, Research Assistant Professor When encountering the Nao robot, Dr. Michael Villano remarks, people behave as if an infant has just been brought into the room. Women are drawn to it as are small children. I ve never seen anyone of any age step away from it. It s such a compelling device, which is what s going to make this study so easy to run. The Notre Dame team s focus in working with the robot is on developing therapeutic protocols and software to support those protocols. They are not interested, says team member Dr. Joshua Diehl, in the commercial potential of such software. We are dedicated only to the science of this research, stresses Dr. Diehl. As a team, we are interested in critically examining the role of the robot in therapy and then publishing our findings. Enhancement of Intervention Dr. Diehl s team is in the early stages of piloting. We ve gone through our first round of piloting in which our aim, in part, was to see how the robot would fit into the therapy session. We wanted to discover what went well, what aspects needed improvement and how the interface was going to function. Now we ve moved on to a development phase during which we re improving the interface s ease of use. We re developing comparison conditions that examine the benefit of an actual robot in the room as opposed to other versions that are less expensive than Nao. We re committed to discovering Nao s best application and, most importantly, to learning which children will benefit most from this robot. Even if it s a given that Nao will be effective for children with autism, it is not yet certain that it will be effective for all children with autism. We want to make certain that we know which kids derive the

3 most benefit from this kind of intervention enhancement, because that is what Nao really is, an enhancement of intervention. Essentially, he explains, ours is an applied behavior analysis therapeutic program into which the robot is integrated in order to provide an enhancement to the existing program. A Fully Functional Humanoid Robot Dr. Charles Crowell, one of the study team s senior researchers, describes the 22-inch humanoid robot that has prompted an unusually positive emotional response from children with autism and their families. Nao s sensory capabilities, Dr. Crowell explains, include cameras that mimic sight; speakers that allow the user to project a synthesized voice; and a microphone that will pick up a voice from interactions. The robot has a touchsensitive area on its head and sonar to detect obstacles to its front and slightly to its side. Nao s motor capabilities, he continues, include motors in its joints that allow it to move legs, knees, arms, shoulders, elbows and to some extent fingers. Nao has three fingers wedded together on each hand and an internal computer that runs a Unix operating system that can serve as a platform for developing and controlling the robot. Nao can communicate via a tethered Ethernet communication line or with Wi-Fi. Nao Simplifies Human Behavior According to Dr. Diehl, the Applied Behavioral Analysis (ABA) sessions they conduct tend to be programmatic and repetitive for good reason: the children need to practice and drill to learn skills. What the robot adds to those sessions is an element that s intrinsically interesting and engaging for the children. It adds spice to the therapy sessions, but Nao s value goes way beyond that. There are many potential benefits to robot use over low-tech, Dr. Diehl insists. Nao allows us to better simulate and simplify human behavior to make that behavior

4 more understandable. It provides a safer environment for the child to practice social skills, like carrying on a conversation. Trying to practice conversation skills with an eight-year-old child with autism and an eight-year-old peer is much more difficult and the setting is less safe, with more potential variables to control, with the therapist not knowing precisely how the peer will react to some situations. Robot use gives us a safe environment so that we can structure a conversation that enables the child to practice these skills multiple times. Behaviors Can Be Developed: Creating a Complex Interface Nao, Dr. Crowell adds, is a complicated device for which behaviors can be developed using techniques to control the motors and to feed the robot information. Performance is induced by a command-driven interaction style or it can be set up to run in an autonomous mode in which its task capabilities are dependent on user inputs. Given that functionality, he says, we believed it was possible to integrate the Nao into a therapeutic setting along with the therapist as a third person in the room who would provide an adjunct to the therapy. We took a simplified approach to the interface creation; baby steps, in fact, Dr. Villano comments. We ve built a Wizard of Oz interface. The program looks much like a universal remote control on a Windows PC. Users can initiate specific Nao behaviors by pushing certain buttons. For instance, a positive feedback behavior will initiate a football referee s touchdown signal by Nao and a voice response that tells a child, way to go, or Nao can assume a generic instructor s stance with hand on hip and a voiceover that tells the child, I m not sure I understood you. There are many other similar behaviors that we can initiate from the universal remote control. Nao Text-to-Speech Capabilities Text-to-speech is another major functionality the team uses to improve interactivity, Dr. Villano says. The vehicle can be as simple as a text box. Anything typed into the text box will be immediately transmitted into the Nao. The Nao will then speak the words that have been typed into the text box by employing its text-to-speech conversation in a humanlike voice. The goal, he adds, is to carry on a conversation but also record pre-set dialogues. Recently, he says, we implemented a way for

5 the Nao to play Simon Says with a child in therapy. Nao can either lead Simon Says or enable the child to lead Simon Says and can participate in the Simon Says sessions by making pre-set movements initiated from this interface. It s amazing how lifelike and yet how unhumanlike these movements can be, Dr. Diehl observes. For example, he adds, we pre-program behaviors through controls so precise that we can subtly manipulate gestures so that each gesture conveys a different meaning, the same way subtle manipulations in our own gestures can change the meaning of what we are trying to communicate. Customizable Capabilities Thanks to the interface, Dr. Villano points out, Nao s capabilities are fully customizable. The behaviors associated with the button labels are driven by a computer configuration. The result is that customized behaviors can be developed in a relatively short time frame and targeted toward a child with whom we re working. For example, he continues, if a child is a fan of the movie Toy Story or enjoys swimming or any other specific activity we can develop some custom behaviors, with companion voiceover, to match those preferences. We have one that says, Hey, why don t you watch me do my backstroke? Then the robot performs backstroke movements. The kids become very excited when they see a robot perform an activity with which they re familiar. The configuration files allow us to customize the activities for individual children. The Project s Genesis: a Shared Interest in Human/Technology Interaction While Dr. Diehl has a layman s interest in technology fueled initially by his younger brother s use of AAC devices, for Dr. Crowell and Dr. Villano technology is a vocation. Their shared interest: how individuals react to technology. The result: the genesis of Notre Dame s Nao project. Says Dr. Crowell, Examination of various forms of human-technology interaction is a major focus of our lab, whether it s humans interacting with computers, humans interacting with virtual environments like video or avatars on a screen, or humans interacting with robots.

6 Several years ago, he recalls, he became interested in robot-human interaction. Dr. Villano and I acquired a conventional robot for our lab. We were attending some conferences where we were trying to present the early studies we conducted with this people-bot robot that doesn t look very humanlike. We had long been interested in acquiring another robot with characteristics that were somewhat more humanlike, because we are very interested in examining the social dynamics between humans and robots, in particular how humans might be influenced by a piece of technology that they perceive to possess humanlike characteristics, whether the technology is an artificially intelligent computer program or a robot with the appearance and actions of a conventional robotic device. We were searching for ways to acquire a more human-looking robot. Then, last year, at a robotics conference, we saw a French company, Aldebaran Robotics ( demonstrate robots named Nao. We were intrigued by the robots capabilities. Dr. Villano and I strategized on the way home from the conference as to how we might be able to assemble the funding in order to acquire one or two of these devices. They eventually obtained funding for two robots. Even before we got them to our lab we understood that children with ASD might possess a special affinity for a Nao device in part because these children tend to be more object-oriented than social. We felt that having these robots in our lab provided us with an opportunity to convince some of our colleagues to collaborate with us to see how the use of these robots in therapy for children with autism might play itself out. That caused us to go next door and find Dr. Diehl and tell him about the Nao. He was very excited about the concept and we began to assemble the current team beyond the three of us. Parents Ask, What Can We Do with This Research? Most of my work has been centered on understanding language and communication difficulties in children with autism, explains Dr. Villano. I heard over and over from parents, There s so much work underway aimed at trying to understand autism, trying to pinpoint its causes, but what can we do with this research? He was already leaning toward therapy intervention work because there was such

7 a need in the community for more knowledge about what works and what doesn t. Nao fit into what I wanted to do in terms of expanding my research in order to provide useful information to the community while also learning more about what we could do to enhance therapies. Before returning to Notre Dame, where he spent his undergraduate years, Dr. Villano spent 15 years in research and development in industry where he sought ways to adopt technologies and technological solutions to business problems, focusing on human-computer interaction and employee training issues. I was actually looking to return to an academic environment and helped create a position for myself at Notre Dame, which led me to resume a collaboration with Dr. Crowell that began when I was an undergrad. Since then Dr. Villano has taught courses on interactions between humans and computers. I ve seen recent research that intrigued me, such as Rosalind Picard s work at MIT ( where technologies were being developed to help children with autism identify other people s expressions or their own expressions. I was impressed with the ways technologies were applied in this area. We had a graduate student in our department then who was developing software that could be utilized with children with autism. The software the student was developing was open source and was intended to be freely available to children and their families. I was impressed with that effort. He began at a low level by engaging other undergrad students to build small games that would be applicable. This spurred discussions between Dr. Crowell and I about other ways in which we could become involved in utilizing technology to impact autism. Will a Robot Ever Be Affordable for Families? Will families be able to afford Nao or a robot that performs similar tasks and offers similar therapeutic opportunities? Says Dr. Diehl, If we get to a point where we determine that there are clear scientifically proven benefits in research, the next question is, What is the best way to integrate this concept in a cost-effective manner?

8 We re already beginning to address that question in the design of our research studies to see if there are less expensive versions of Nao that provide equal benefit. Nevertheless, in an economic crisis, affording Nao is difficult. If Nao works, how can we make it available? Perhaps, he suggests, through a recorded video screen program that a therapist could use. Maybe we don t even need a robot. Maybe there s an animation program that is just as effective. The trouble is, we don t have that information yet. There s something else of interest to Dr. Crowell and me, Dr. Villano declares. We are already beginning to look at alternatives to Nao. Nao is a great research and development platform that was specifically designed for an academic setting. There are many low-cost alternative robots that we might be able to manage. First, we need to identify the Nao features that are most engaging and that exert the most impact on the therapy. Perhaps, for example, there s a device that possesses Nao s physical movements but those movements aren t as graceful as Nao s. There might be less expensive but effective robots that cost under $1,000 as opposed to Nao s nearly $20,000 price tag. This would be far more affordable for therapists to use for multiple children and might be more affordable for families as well. As Dr. Diehl says, super low-cost alternatives would provide therapists and families with a piece of software with animation, like a video game. Maybe there s an aspect of such an advanced robot like Nao that may prove critical. Maybe that high level of technology is what s needed. At that point the question is, How do we make this technology more available? With tongue only half embedded in check, Dr. Crowell replies, We may take Dr. Diehl to France to persuade the Nao manufacturer to produce a lower cost robot specifically for this purpose. Dr. Villano responds, If the company can be convinced that it ll get 100,000 orders instead of 1,000 orders I m sure it could lower the cost significantly. Robotic Applications for Other Children with Disabilities

Would robotic applications prove appropriate for other children with disabilities beyond those with ASD? According to Dr. Diehl, the simple answer is yes.

9 Would robotic applications prove appropriate for other children with disabilities beyond those with ASD? According to Dr. Diehl, the simple answer is yes. If we see the science playing towards beneficial response from kids with autism, we can say, It was easy for us to make the connection between children with autism and robots or technology in general. It s not that big a step to reasonably hope that other children with developmental disabilities, especially those with social interaction, are going to respond it similar ways. That would be a next application. The programs may differ from those used for children with ASD, but that s something for research to point the way to. The Team Approach: How It Works For Dr. Diehl, a team approach to research is the only approach he s known during his professional career. The usual barriers in any team effort include the inability to gather all team members for meetings, organizational issues and the struggle to constantly make certain that all team members are on the same page. Fortunately, in this project we ve encountered very few of the usual potholes, probably because most of us are quite experienced in team situations. This is a project I could never have come close to handling alone, he admits. This team has so many individuals with specific areas of expertise. Each of us has his or her own piece. It s not like this project is one individual s private preserve and the other team members are only assisting. Each of us has our own areas of expertise, from the primary researchers to the therapists to the expertise contributed by the many students who are helping out. There is no downside to this effort. Coming from industry, says Dr. Villano, I ve been on highly functional teams and teams that are highly dysfunctional. This team works extraordinarily well. We have so many skill sets that complement each other. There s a tremendous amount of mutual respect and lack of ego, which makes it a pleasure to get together. Nao Training for Children and Therapists: Control by ipad? Nao training for children is very basic, Dr. Diehl says. All that s necessary is for children to know the ground rules

10 before their ABA session. Hopefully, he adds, those rules are no different than for any ABA session other than there is a robot involved. For example, one of the ground rules would be that the child cannot pick up and throw the robot. Our goal is to integrate the robot as seamlessly as possible into the existing and widely used behavior therapies for children. For the therapist, he notes, robot control in the same room as the session is our aspiration, replacing the current wizard located in a room adjoining the therapy session. Eventually we d like to take that interface and combine it with a mechanism that can be controlled by the therapist during sessions by hand via an ipad-like device. This would enable the therapist to have full control for the integration of the device into the therapy session. We want to make that interface sufficiently simple and adaptable so that only minimal training would be required for a therapist with a basic knowledge of computers to control this robot. The ultimate goal is to make the device as simple and straightforward as possible. Cool Steps: An Enhancement Wish List Beyond therapist control, Dr. Crowell says, there are several cool steps we can imagine, one of which might be that the Nao doesn t necessarily have to be physically present during therapy sessions. To replace Nao s physical presence, he points out, all that would be needed is a video of the Nao that the therapist can arrange to behave like the real Nao by invoking pre-recorded sequences. We might be able to create an ipad-like tool enabling the therapist to call up the Nao in a virtual way and get the Nao to behave like the current Nao because we would have learned that Nao s behaviors were the most effective methods of interaction with the child. That s just one of the many possible next steps. Dr. Villano cites Dr. Crowell s observation that Nao possesses many sensing capabilities. One of the problems I see is that many people try to make the robots autonomous or semi-autonomous. There s an awful lot of effort expended in time and research trying to determine how to get the robot to understand the voice of the child, what the child is saying, or react to movements the child might make by sensing its movement or doing face recognition. There is a gamut of artificial intelligence or computer science-based applications aimed at producing a robot that is not human-controlled but instead robot-

11 controlled in that the robot interacts directly with the child. For me that is a very difficult research path. Many promises have been made over the past years since he was in graduate school that have not been satisfactorily fulfilled, Dr. Villano declares. In our research we didn t experiment with an autonomous or semi-autonomous Nao, but in terms of possible future research directions it would be beneficial if aspects that were successful in robot autonomy research could be implemented. For example, he says, it would be very helpful if a robot could track a child s face. The Nao does not currently perform that function and performing it would make the robot experience far more compelling for a child with autism. There are similar approaches we might adopt in order to create a more integrated experience thereby turning over a little control to the robot itself to see if that would make the robot more engaging and even more useful in a therapy session. Personally, he adds, I suspect that it is the physical embodiment of the robot in the room that creates the most excitement for children. It s one thing to see an image of a robot on a computer screen children are accustomed to that but quite another for kids who have never seen a robot live. The children have an incredible reaction to the robot in the room. Dr. Villano and Dr. Crowell continue to utilize an industrial Peoplebot ( robot in some of their research. It resembles the robots that are visible traveling through hospital corridors. It s a tall open metal cylinder. We ran studies using that robot. Some people jumped away from it. They were frightened when it came into the room. Women responded to it very differently than did the men. Translating Research into Fun for Kids and Families I said this to some students recently when I was speaking to them about careers in psychology: Of all the projects I ve been on this is the most fun and rewarding. I ve worked on seven-year development efforts with a core team of people in my corporate life but there is something about doing something for other people and watching technology benefit the greater good that produces a very different

12 level of satisfaction, which is a major aspect of Notre Dame s academic mission and one that all three of us buy into and to which we are deeply committed. Dr. Crowell attributes much of the team s success so far to my young colleague, Dr. Diehl. Of all the people who could make or break the project he is in a position to do it. He has responded not only with a tremendous amount of enthusiasm and energy but also humility. He doesn t care who gets the credit. He s focused on getting the project done and is committed to the research. When you have an individual like that who is the lead person on a team with recognition and expertise in this area we can t help but be a success. So he s going to make the rest of us look good. Concludes Dr. Diehl, We re having a lot of fun and really enjoying this experience. Now the goal is to translate our enjoyment into fun for the kids and their families.

Reflection Guide for Interns

Reflection Guide for Interns As an intern, your supervisor may ask you to reflect on your development and accomplishments, in order to enhance the educational experience and help you communicate what you have learned to future employers.