A Virtual World for an Autonomous Underwater Vehicle. Final Proposal for The Edge, SIGGRAPH 94

Transcription

1 A Virtual World for an Autonomous Underwater Vehicle Final Proposal for The Edge, SIGGRAPH 94 CONTACT Primary: Don Brutzman Code OR/Br, Naval Postgraduate School Monterey CA (408) office, (408) home (408) FAX, Collaborators: Dr. Michael J. Zyda Michael Macedonia Rodney Luck CATEGORIES Primary: Scientific, Academic, Educational Secondary: Commercial, Government, Communication, Simulation & Training CATALOG Project Name: An Underwater Virtual World for an Autonomous Underwater Vehicle Media: Focus: Description: 3D real-time computer graphics, video, data sonification, poster Sonar and underwater terrain visualization Virtual world environment Underwater robot operating in-the-loop Global collaboration and visual output using Multicast BackbONE (MBONE) and Mosaic It is tremendously difficult to observe and test underwater robots, because they operate in a remote and hazardous environment where physical dynamics and sensing modalities are counterintuitive. An underwater virtual world can comprehensively model all salient functional characteristics of the real world in real time. This underwater virtual world is designed from the perspective of an autonomous underwater vehicle (AUV), enabling realistic AUV evaluation and testing in the laboratory. Scientific visualization and 3D real-time graphics enables local and remote researchers to observe robot interactions within a full model of Monterey Bay. The Edge SIGGRAPH 94: A Virtual World for an Autonomous Underwater Vehicle page 1

2 Hardware: Software: Slides and video: SGI Onyx Reality Engine (4 processors) SGI Indigo Extreme (second window into world) SGI Indigo Extreme (MBONE/Mosaic video/audio/etc.) T1 Ethernet link to Internet with multicast support (may require an additional multicast router) Big-screen television monitor (70" diagonal) Networked robot processors (Gespac 68030/80486) VCR player and television monitor Developed for Ph.D. dissertation in computer science. Underwater virtual world code written in C++ and C, with graphics code utilizing SGI Inventor applications programming interface. Network software includes standard Ethernet sockets, multicast Distributed Interactive Simulation (DIS) v protocol packets and publicly-available MBONE tools for Internet-wide video, audio and image transmission. Robot software is written in C and Ada, running under OS-9 real-time operating system and DR-DOS 6. All source code, images and pertinent papers will be available without charge via anonymous ftp. Slides mailed separately, video of current development status also submitted. EXHIBIT Space requirements: (height width depth) Big screen TV 5 x 4 x 2 movable Graphics workstations 4 x 6 x 3 table Robot hardware 5 x 5 x 3 cart VCR display 6 x 2 x 2 stand Lighting: Power: Audio/Acoustics: No special requirements. Standard power requirements for SGI workstations listed: 20 amps service for the Onyx and 15 amps each for the two Indigos. Standard 120V to run big-screen TV, accompanying VCR and equivalent of 2 PC s for the robot cart. If a multicast router is needed add 5 amps at 120V. No other special requirements. Sound samples are generated by workstations during interaction with the virtual world. Additionally the VCR display will put out audio while playing prerecorded video clips. No special support is needed. Capacity: Length of Experience: 2-5 minutes Viewers per experience: 1-8 The Edge SIGGRAPH 94: A Virtual World for an Autonomous Underwater Vehicle page 2

3 The autonomous robot will be navigating submerged within a simulated Monterey Bay using sonar sensor and terrain models. By design the mission is automatically controlled by robot interactions within the synthetic environment. Event capacity is prepared to vary if necessary to match audience response. Furniture: Graphics workstations table 4 x 6 x 3 VCR display stand 6 x 2 x 2 Chairs 3 Set-up and strike times: Personnel required: Estimated shipping costs: 6 hours 1 minimum, 2 average throughout. Monterey CA to/from SIGGRAPH Orlando FL. Based on recent shipment to East coast, estimated total shipping cost is $4000. Achieving a lower cost will depend on whether any SGI workstations and TV display monitors can be provided by SIGGRAPH 94 or rented locally. Current funding for this proposal is uncertain but possible. Please advise regarding possible support. The Edge SIGGRAPH 94: A Virtual World for an Autonomous Underwater Vehicle page 3

4 Floor plan: Figure 1. Floor plan for underwater virtual world exhibit at The Edge, SIGGRAPH 94. The Edge SIGGRAPH 94: A Virtual World for an Autonomous Underwater Vehicle page 4

5 ABSTRACT A critical bottleneck exists in Autonomous Underwater Vehicle (AUV) design and development. It is tremendously difficult to observe, communicate with and test underwater robots, because they operate in a remote and hazardous environment where physical dynamics and sensing modalities are counterintuitive. An underwater virtual world can comprehensively model all salient functional characteristics of the real world in real time. This virtual world is designed from the perspective of the robot, enabling realistic AUV evaluation and testing in the laboratory. 3D real-time graphics are our window into that virtual world. Visualization of robot interactions within a virtual world permits sophisticated analyses of robot performance that are otherwise unavailable. Sonar visualization permits researchers to accurately "look over the robot s shoulder" or even "see through the robot s eyes" to intuitively understand sensor-environment interactions. Distribution of underwater virtual world components enables scalability and real-time response. The IEEE Distributed Interactive Simulation (DIS) protocol is used for compatible live interaction with other virtual worlds. Network access allows individuals remote access. This is demonstrated via MBONE collaboration with others outside The Edge, and Mosaic access to pertinent archived images, papers, datasets, software, sound clips, text and any other computerstorable media. This project presents the frontier of 3D real-time graphics for underwater robotics, ocean exploration, sonar visualization and worldwide scientific collaboration. The Edge SIGGRAPH 94: A Virtual World for an Autonomous Underwater Vehicle page 5

6 FULL DESCRIPTION The problem. A critical bottleneck exists in Autonomous Underwater Vehicle (AUV) design and development. It is tremendously difficult to observe, communicate with and test underwater robots, because they operate in a remote and hazardous environment where physical dynamics and sensing modalities are counterintuitive. The solution. An underwater virtual world can comprehensively model all salient functional characteristics of the real world in real time. This virtual world is designed from the perspective of the robot, enabling realistic AUV evaluation and testing in the laboratory. 3D real-time graphics are our window into the virtual world. AUV development difficulties. The primary difficulty facing AUV developers is a challenging physical environment: an operating AUV is inaccessible, remote, and unattended. It is subjected to extremes of pressure, temperature, corrosion. Communications are intermittent or nonexistent. Sonar sensing is slower and very much different from vision. Vehicle deployment, operation and recovery are time-consuming and expensive. Vehicle physical dynamics control is very challenging. There are six spatial degrees of freedom (position and rotation), not all physical control issues are solved, and there may be an unpredictable influence by currents. Propulsion is costly, slow and limited. A typical vehicle only has a few hours endurance. There is clear empirical evidence of a severe bottleneck in underwater robotics. There are thousands of indoor and outdoor land-based mobile robots, many hundreds of airborne and space-based autonomous robots, and many hundreds of underwater remotely operated vehicles (ROVs). In contrast there are perhaps a dozen working AUVs in existence, each with limited functionality. AUV failure in the ocean is unacceptable for several reasons: any failure may become catastrophic, recovery may be difficult or pointless, and replacement costs in time and money are prohibitive. We can conclude the following: reliability, stability and autonomy are paramount, AUV constraints are often worst-case for any type of robot due to underwater environment, and many theoretical and engineering problems remain open. Why an underwater virtual world? "Virtual world system..characteristics are seeing and interacting with distant, expensive, hazardous, or non-existent 3D environments. The technology for seeing is real-time, interactive 3D computer graphics and the technology for interacting is evolving and varied." (Zyda 1992) The current underwater robot development paradigm is inadequate and costly. Piecemeal design verification and individual component simulations are not adequate to develop and evaluate artificial intelligence (AI)-based robot The Edge SIGGRAPH 94: A Virtual World for an Autonomous Underwater Vehicle page 6

7 systems. Virtual world systems provide a capability for people or robots to see and interact within synthetic environments. The research goal is to provide complete functionality of the target environment in the lab, providing adequate simulation scope and interaction capability to overcome the inherent design handicaps of classical simulation approaches. AUV underwater virtual worlds may break the AUV development bottleneck. AUV underwater virtual world characteristics. The underwater virtual world must recreate the complete environment external to the robot. Robot sensors and analog devices must be modeled accurately. Robot physical dynamics behavior must be adequately simulated, since underwater vehicles are prone to non-linear dynamic instabilities and unpredicted physical responses may result in vehicle loss. To minimize sources of simulation error, an exact copy of robot hardware and software is plugged into the virtual world using physical or logical sensor and actuator connections. The difference between operation in a virtual world or an actual environment must be transparent to the robot. Finally, successful implementation of a virtual world can be validated by identical robot performance in each domain. This is a type of Turing test from the robot s perspective: if robot performance is identical in each domain, then the virtual world is functionally equivalent to the real world. Importance of sensors. Design of autonomous underwater robots is particularly difficult due to the physical and sensor challenges of the underwater environment. Robot performance is often very tightly coupled to sensor accuracy and interpretation. Emergent behavior from interaction between robot processes and the environment can only be determined through experimentation. Having valid sonar and terrain models is very valuable for robot design and testing since sensor interactions are repeatable indefinitely. This means that machine learning based on massive repetitive training is feasible, such as training genetic algorithms or neural networks. Additionally, potentially fatal scenarios can be attempted repeatedly without risk to robot, human or environment until success is reliably achieved. Sonar visualization. Visualization of robot sensor interactions within a virtual world permits sophisticated analyses of robot performance that are otherwise unavailable. Sonar visualization permits researchers to accurately "look" over the robot s shoulder or even "see" through the robot s eyes to intuitively understand sensor-environment interactions. Similar in-depth analysis is not possible using traditional test methods such as individual software module evaluation, direct robot observation or post-mission reconstruction. In particular, the overwhelming size and information content of ocean and robot datasets means that visualization is essential to The Edge SIGGRAPH 94: A Virtual World for an Autonomous Underwater Vehicle page 7

8 extract meaning from numerous simultaneous quantitative relationships. Visualization of the robot in its surroundings greatly improves human understanding. Networking. Distribution of underwater virtual world components enables scalability and real-time response. The IEEE Distributed Interactive Simulation (DIS) version protocol is used for compatible interaction with other virtual worlds listening on the Internet. A distributed approach also minimizes dependence on unique (or hard-to-replace) hardware and software. This project is an excellent application to take advantage of a high-bandwidth information superhighway, further extending the capabilities of multiple researchers. The network approach allows many individuals dynamic remote access, which is demonstrated by MBONE transmission of video, graphics, and sound for collaboration with other participants outside The Edge. Providing hypermedia access via publicly available World-Wide-Web network browsers such as Mosaic makes a complete variety of pertinent archived information available to anyone, including images, papers, datasets, software, sound clips, text and any other computer-storable media. Together MBONE and Mosaic are today s "killer applications" on the information superhighway, letting anyone watch and listen in your work. Addition of multicast networked DIS packets and publicly available software lets people observe an identical interactive virtual world locally. Remote interaction within the virtual world is feasible but will be restricted during this demonstration to allow participants in The Edge to view reliable, prepared and information-intensive scenarios. A new paradigm shift in the making? Within two lifetimes we have seen major paradigm shifts in the ways that people record information. Handwriting gave way to typing, and then typing to word processing. It was only a short while afterwards that preparing text with graphic images was easily accessible, enabling individuals to perform desktop publishing. Currently people can use 3D real-time interactive graphics simulations and dynamic "documents" with multimedia hooks to record and communicate information. In this project we see a further paradigm shift becoming possible for researchers. The long-term potential of virtual worlds is to serve as an archive and interaction medium, combining massive and dissimilar datasets and datastreams of every conceivable type. Virtual worlds will then enable comprehensive and consistent interaction by humans, robots and software agents within those massive datasets, datastreams and models that recreate reality. Research conclusions. Construction of an underwater virtual world is feasible. Using 3D real-time graphics in an underwater virtual world enables effective AUV development. The Edge SIGGRAPH 94: A Virtual World for an Autonomous Underwater Vehicle page 8

9 Visualization of robot interactions in an underwater virtual world improves our perceptual ability to evaluate robot performance. A networked robot and virtual world makes robotics research and collaboration accessible worldwide. Attendee experiences. Length of a typical experience with the underwater virtual world is 2-5 minutes, with 1-8 viewers possible per session. This gives a projected capacity of about 200 people per hour. The autonomous robot will be navigating submerged within a simulated Monterey Bay using sonar sensor and terrain models. By design the mission is automatically controlled by robot interactions within the synthetic environment. Demonstrators will answer viewer questions and guide viewer interaction without being distracted by "controlling" the action. Since the underwater virtual world is built primarily to serve the robot, user actions typically modify the 3d real-time graphics "windows" into this virtual world. Permitted user interactions will include changing visualization perspectives, sending commands to the robot and interacting with remote MBONE viewers. When operating at depth, live video from an actual remote operated vehicle (ROV) beneath Monterey Bay will be provided from the Monterey Bay Aquarium Research Institute (MBARI) via MBONE. A poster and a tape loop running on the VCR will give repeating detailed summaries of this project. Viewers can choose to watch the underwater virtual world for either a short time or a long time. Depending on their interests, viewers can interact via the graphics workstations, watch the big graphics monitor, or watch the VCR TV program. Event capacity may vary according to audience response. We will be prepared to turn the "level of detail" knob left or right to best serve individuals in the audience. Demonstration Concerns. This proposal is intentionally broad to demonstrate integration of a robot into a virtual world as well as a new collaboration paradigm. Particular attention will be paid during preparation to ensure reliability of all parts in a demonstration environment. Over ninety percent of the many software components described currently exist and half have been integrated within the virtual world framework. A series of group demonstrations will be conducted locally and over the MBONE prior to The Edge to ensure that the underwater virtual world is "ready for prime time." Bandwidth Budget. I hope that an average bandwidth of 325 kilo bits per second (Kbps) is available (i.e. 25% of a 1.5 Mbps T1 line). This bandwidth budget includes 128 Kbps for locally generated video/graphics, 75 Kbps for a shared audio channel and 128 Kbps for receiving the video feed from the MBARI ROV operating in Monterey Bay. 128 Kbps is the default bandwidth for world-wide multicast video programs and equates to 3-4 frames per second. Lower The Edge SIGGRAPH 94: A Virtual World for an Autonomous Underwater Vehicle page 9

10 bandwidths and a corresponding drop in frame rate are feasible. DIS protocol bandwidth will be negligible. Mosaic access to archived datasets will not be a local load since they will remain available to outsiders only via the NPS anonymous ftp site in Monterey. I am prepared to coordinate directly with local network providers to ensure that Internet multicast support preparations are adequate. If I need to bring a multicast router (mrouter) of course it will make multicast available to anyone on the SIGGRAPH subnet. Relation to The Edge. It is exciting to see so many areas of overlap between this project and the objectives of The Edge. Primary areas of shared interest are scientific, academic and educational for contributions to underwater robotics research and sonar visualization. Secondary categories regarding commercial, government, communication, simulation and training are pertinent due to the high potential value of both underwater robots and underwater virtual worlds. The Edge is intended to include shared experiences, simulation, training, education, virtual environments, high-bandwidth networked graphics, telepresence and telerobotics; this underwater virtual world has components and impact in each of these areas. We expect to inspire and stimulate attendees to consider a myriad of opportunities previously considered inaccessible. In return from those attendees we expect to gain even more insight into the potential and execution of underwater virtual worlds. This project presents the frontier of 3D real-time graphics for underwater robotics, ocean exploration, sonar visualization and worldwide scientific collaboration. We look forward to presenting at SIGGRAPH 94. The Edge SIGGRAPH 94: A Virtual World for an Autonomous Underwater Vehicle page 10

11 SLIDES Figure 2. Figure 3. Figure 4. Figure 5. Figure 6. Figure 7. Figure 8. Naval Postgraduate School (NPS) Autonomous Underwater Vehicle (AUV) graphic rendering using SGI Inventor toolkit. NPS AUV wireframe view showing internal components and Inventor toolkit flexibility. Multicast BackbONE (MBONE) tools showing video, audio and whiteboard tools receiving a live broadcast. This session shows remote operated vehicle (ROV) Ventana just prior to launch and control by Monterey Bay Aquarium Research Institute (MBARI) scientists. Live video and audio links from Monterey Bay are possible daily via microwave relay between support ship RV Point Sur and shore facilities. Example minefield search mission. The AUV must detect, map and avoid mines in a multi-level minefield. This is likely to be one of the sample missions performed during The Edge. Example rendering of actual sound speed profile data sampled in Monterey Bay to visually show predicted sonar refraction and performance. Inventor model of Woods Hole Oceanographic Institution ROV JASON. This model was provided by (in Inventor.iv format) and used without modification. It is included to show compatibility with other graphics models in use. Closeup of stereo imagery from an underwater video camera mounted on the MBARI ROV to show typical MBONE video clarity. The Edge SIGGRAPH 94: A Virtual World for an Autonomous Underwater Vehicle page 11

12 VIDEO 0:00 Demonstration of sample 3D graphics using Inventor. Current work in progress includes addition of DIS read routines to link vehicle postures and articulated components to the DIS packets produced by the (already running) six degree of freedom hydrodynamics model, which in turn is interacting with the autonomous underwater vehicle. Many more Inventor graphics objects will populate the underwater virtual world, include a terrain model produced from actual bathymetry of Monterey Bay. 2:10 Rendering of environmental Sound Speed Profile (SSP) used in sonar model. Sonar performance is highly dependent on SSP. Visualization techniques are employed to aid in understanding the precise nature of robot sonar sensing and how it is affected by the environment. 1:45 Mosaic archive is currently available (see References for URL). 2:10 MBONE tools screen snapshot and network map of regional collaboration sites. 2:30 More SSP plots and renderings. Representative SSP data is available from over a dozen sample points in Monterey Bay taken at three month intervals to reveal seasonal variations. 2:40 Pool traversal and wall detection by an AUV - sample mission showing playback of collected telemetry data. 4:40 Mosaic display of contact information is standard on the World Wide Web. 5:00 NPS AUV summary tape from IEEE Robotics & Automation 92 (see references). 7:45 Local KSBW-TV spot on the initial use of MBONE at NPS, aired June 30, Somehow we managed to keep their facts correct but only most of the time! To date no one has performed experiments with the space shuttle via MBONE (although it is technically feasible). 10:00 total This video is preliminary. A more detailed and polished video will be produced during preparation demonstrations, and will be displayed during The Edge as part of the VCR tape loop. The Edge SIGGRAPH 94: A Virtual World for an Autonomous Underwater Vehicle page 12

13 REFERENCES Brutzman, Donald P "From virtual world to reality: designing an autonomous underwater robot," American Association for Artificial Intelligence (AAAI) Fall Symposium on Applications of Artificial Intelligence to Real-World Autonomous Mobile Robots, Cambridge, Massachusetts, October 23-25, 1992, pp Macedonia, Michael R. and Brutzman, Donald P "MBONE Provides Video and Audio Across the Internet," IEEE Computer, to appear April Brutzman, Donald P., Kanayama, Yutaka, and Zyda, Michael J "Integrated Simulation for Rapid Development of Autonomous Underwater Vehicles", Proceedings of the IEEE Oceanic Engineering Society Conference AUV 92, Washington DC, June 2-3, 1992, pp Brutzman, Donald P. and Compton, Mark A "AUV Research at the Naval Postgraduate School," Sea Technology, vol. 32 no. 12, December 1991, pp Brutzman, Donald P NPS AUV Integrated Simulator, Master s Thesis, Naval Postgraduate School, Monterey, California, March Brutzman, Donald P., Floyd, Charles A. and Whalen, Russell "Naval Postgraduate School Autonomous Underwater Vehicle," Video Proceedings of the IEEE International Conference on Robotics and Automation 92, Nice, France, May Brutzman, Donald P., editor Video Proceedings of the Eighth International Symposium on Unmanned Untethered Submersible Technology, University of New Hampshire, Durham, New Hampshire, September 27-29, Ziomek, Lawrence J. and Policky, F. Wynn. "The RRA Algorithm: Recursive Ray Acoustics for Three-Dimensional Speeds of Sound," IEEE Journal of Oceanic Engineering, vol. 18 no. 1, January 1993, pp Zyda, Michael J., Jurewicz, Thomas A., Floyd, Charles A. and McGhee, Robert B "Physically Based Modeling of Rigid Body Motion in a Real-Time Graphical Simulator," unpublished paper, Naval Postgraduate School, Monterey, California, September Zyda, Michael J., Pratt, David R., Falby, John S. and Kelleher, Kristen M "The Software Required for the Computer Generation of Virtual Environments," Presence: Teleoperators and Virtual Environments, vol. 2 no. 2, Spring 1993, pp (Reference copies available upon request or via anonymous ftp, see NPS Mosaic home page file://taurus.cs.nps.navy.mil/pub/mosaic/nps_mosaic.html for details.) The Edge SIGGRAPH 94: A Virtual World for an Autonomous Underwater Vehicle page 13

14 AUTHORIZATION Permission to Use Visual and Audio: Conference Presentation Release: In the event that materials used in SIGGRAPH The Edge entry contain the work of other individuals or organizations (including any copyrighted musical compositions or excerpts thereof), I understand that it is my responsibility to secure any necessary permissions and/or licenses. I have the necessary rights and/or permissions to use the visuals in my piece. I have the necessary rights and/or permissions to use the audio in my piece. By my signature below, I grant SIGGRAPH 94 permission to use my The Edge entry in The Edge venue. I maintain the copyright to my work and will receive full credit wherever this work is used. Distribution Rights: I grant non-exclusive worldwide distribution rights to ACM SIGGRAPH to publish and distribute my slides, films and videos in the following venues. I maintain the copyright to my work and will receive full credit wherever my image is used: The SIGGRAPH 94 Visual Proceedings The SIGGRAPH 94 Multimedia CD-ROM The SIGGRAPH 94 Video Review (ACM SIGGRAPH s video publication) Conference Promotional Material: I grant ACM SIGGRAPH the right to use my slides, films and videos for conference and organization publicity, both now and in the future. This includes usage on posters, brochures, catalogs, coffee mugs, or media broadcast. In exchange, SIGGRAPH provides full author/artist credit information. I grant ACM SIGGRAPH permission to use my work for conference and organization publicity. Signature: March 8, 1994 Donald P. Brutzman The Edge SIGGRAPH 94: A Virtual World for an Autonomous Underwater Vehicle page 14