Developing Virtual Reality Applications

Transcription

1 Developing Virtual Reality Applications

2

3 Developing Virtual Reality Applications Foundations of Effective Design Alan B. Craig William R. Sherman Jeffrey D. Will AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD PARIS SAN DIEGO SAN FRANCISCO SINGAPORE SYDNEY TOKYO Morgan Kaufmann Publishers is an imprint of Elsevier

4 Morgan Kaufmann Publishers is an imprint of Elsevier. 30 Corporate Drive, Suite 400, Burlington, MA 01803, USA This book is printed on acid-free paper by Elsevier Inc. All rights reserved. Designations used by companies to distinguish their products are often claimed as trademarks or registered trademarks. In all instances in which Morgan Kaufmann Publishers is aware of a claim, the product names appear in initial capital or all capital letters. All trademarks that appear or are otherwise referred to in this work belong to their respective owners. Neither Morgan Kaufmann Publishers nor the authors and other contributors of this work have any relationship or affiliation with such trademark owners nor do such trademark owners confirm, endorse or approve the contents of this work. Readers, however, should contact the appropriate companies for more information regarding trademarks and any related registrations. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means electronic, mechanical, photocopying, scanning, or otherwise without prior written permission of the publisher. Permissions may be sought directly from Elsevier s Science & Technology Rights Department in Oxford, UK: phone: ( 44) , fax: ( 44) , permissions@elsevier.com. You may also complete your request online via the Elsevier homepage ( ), by selecting Support & Contact then Copyright and Permission and then Obtaining Permissions. Library of Congress Cataloging-in-Publication Data Application submitted British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library. ISBN : For information on all Morgan Kaufmann publications, visit our Web site at or Printed in China

5 Dedicated to Colleen and Cara, the future is yours Alan Dedicated to Anthony, Thomas, Theresa, Danielle, Cindy, and Sheryl for their love and support throughout the years Bill Dedicated to the Boy, the Girl, and the Mother thereof Jeff

6

7 Contents PREFACE... xi ACKNOWLEDGMENTS...xv CHAPTER 1 Introduction to Virtual Reality What is virtual reality? The beginnings of VR Morton Heilig s Sensorama Ivan Sutherland s vision for computer-based virtual reality Myron Krueger s videoplace University of North Carolina at Chapel Hill Electronic Visualization Lab at the University of Illinois at Chicago VR paradigms Collaboration Virtual reality systems Hardware Software Representation User interaction Interaction Techniques Making selections Manipulating the virtual world Navigation...30 CHAPTER 2 Applying Virtual Reality Virtual reality: the medium Form and genre What makes an application a good candidate for VR Promising application fi elds Virtual prototyping Architectural walkthroughs...37 vii

8 viii Contents Visualization Training Entertainment Other application genres Demonstrated benefi ts of virtual reality More recent trends in virtual reality application development Converting extant applications to virtual reality without having access to the application code Developing virtual reality applications with game engines Low-cost input devices Cluster-based compute engines and high-performance graphics acceleration Passive stereo displays Augmented reality and handheld devices Wireless interaction and optical tracking More senses Multiperson virtual and augmented reality Tiled displays Head-based projection Tele-immersion A framework for VR application development...54 About the applications in this book...57 CHAPTER 3 Business and Manufacturing Areas of application Product development Business education Marketing Other and future usage of VR in business and manufacturing...63 CHAPTER 4 Science Applications Areas of application Exploration Physical system simulation and interaction Areas of application in science CHAPTER 5 Medical Applications Areas of application...146

9 Contents ix CHAPTER 6 Education Applications Areas of application Other and future uses CHAPTER 7 Public Safety and Military Applications Areas of application Equipment operation training CHAPTER 8 Art Areas of application Design and sketching Exploration of the medium Empathetic experiences Exploration of the human condition Other and future usage of VR in art CHAPTER 9 Entertainment Applications Areas of application Games Interactive fi ction Creative expression Entertainment production CHAPTER 10 Putting It All Together Executive summary Introduction The database fi elds Finding applications with specifi c characteristics Showing all the characteristics of a specifi c application Application taxonomies BIBLIOGRAPHY INDEX

10

11 Preface HOW THIS BOOK CAME TO BE This book, Developing Virtual Reality Applications: Foundations of Effective Design, has been many years in the making. Our interest in virtual reality came about as an outgrowth of our interest in, and our day-to-day work in scientific visualization. In the early 1990s the state-of-the-art in computer graphics used for scientific visualization was to take scientific data, clean it up, create geometry from it, place computer graphics lights, choose a camera perspective, and then render, not in real time, single images of computer graphics output. To create animations, one then created a sequence of images where either the camera perspective, or the underlying data evolves over time. The resulting images could be recorded to video tape or film, and played back as an animation. There are several obvious drawbacks in that scenario. A fairly typical rendering time was about 20 minutes per frame. As technology improved, and rendering rates increased, frames still typically took about 20 minutes to render, because designers chose to render frames of greater complexity. Thus, scientific visualization animations were primarily used as an explanatory tool, rather than as an interactive, exploratory, real-time mechanism for exploring data. The use of computer graphics in other areas, such as Computer Aided Design, animated movies, architecture, and others followed a similar course of development. The obvious thing that was missing was the human in the loop. The improvement in computer hardware and software allowed pseudo real-time computer graphics to be produced. This allowed new imagery to be generated as time evolved in the underlying data and/or the viewer changed their physical perspective to see the imagery from a different point of view. Through our work with academic, government, and corporate partners, we began to see numerous areas where real-time imagery, in the form of virtual reality, could be applied with great benefit. A crystallizing application was when xi

12 xii Preface we were doing visibility studies to determine operator visibility from within the cab of a backhoe loader, and were doing animations to show what the operator could see from the vantage point within the cab. However, those studies were not realistic in that in the real world, an operator could move his head in order to see around obstacles. This application (described in Chapter 3 of this book) provided the initial funding to develop our virtual reality lab. As we developed the VR lab and started developing applications, we were invited to visit other VR labs and had a chance to experience and analyze the applications they had developed. Our home research lab at the University of Illinois was host to a plethora of visitors from academia, government, and industry. Visitors to the National Center for Supercomputing Applications at Illinois frequently requested a visit to the VR lab. As we met these visitors, we were constantly asked for information related to what applications had been developed in their area of interest. As such we became brokers between people looking for information, and those who were actively seeking information regarding VR applications. Additionally, because of the visitors to our lab and our visits to other labs, we were able to see many ideas and trends about what was working well, and what was not working well, and general observations across a vast array of VR applications. Visitors were constantly asking us to write this information down, and that if we did, that they wanted the very first copy. So, in essence, we wrote this book to satisfy the requests of many people who were seeking information of the very type that is included in this book. When we met Mike Morgan of Morgan Kaufmann Publishing, he was looking for authors to write a book about scientific visualization. As we discussed that with him, we brought up the idea of a book that covers numerous VR applications in depth, in a way that could help VR developers learn from the mistakes of others, as well as to learn about ideas that worked particularly well. Simultaneously, the book would be useful to an audience of people who are interested in how VR can be applied in their area of interest. Morgan was enthused by that idea, and we wrote this book to fulfill that dream. Interestingly, we felt it important to have an introductory chapter to provide background to enable readers to understand the concepts and technologies we describe in the application chapters. That chapter grew to the point where it merited a book of its own, and it lead to the publication of Understanding Virtual Reality: Interface, Application, and Design by William R. Sherman and Alan B. Craig. Though the book you are reading has an introductory chapter that is sufficiently comprehensive to provide the background necessary to understand the application write-ups, the reader who is thirsting for a more in-depth treatment of VR is directed toward Understanding Virtual Reality. WHO THIS BOOK IS FOR This book was created as a resource for several different categories of readers. Virtual Reality application developers will find a wealth of ideas that are directly applicable to their own creations. By standing on the shoulders of giants, the developer can benefit from the trials, errors, and successes of other VR development projects. Corporate research directors who are interested in how VR might (or might not) benefit them in their own companies will find

13 Preface xiii applications that are directly pertinent to their interests, as well as other applications that might not have crossed their minds, yet are likely to be of interest to them. For example, the manager who is interested in using VR for virtual prototyping may not have thought of the potential for using VR as part of an advertising campaign for their product. University students will find this book to be of considerable interest if their field is related to any sort of representation of information. We believe this book will be of interest to students of computer science, psychology, media studies, engineering, and many other disciplines. With the popularity of online virtual worlds, there are a number of people including teachers, students, and the general public who are interested in how online virtual worlds can be taken to the next level and implemented as full blown virtual reality applications. Likewise, this book can provide inspiration and insights for computer game developers. WHAT THIS BOOK IS NOT This book is not a book about computer graphics. There are a vast number of very good texts that cover the details of computer graphics. Likewise, this book is not meant to teach you how to write the computer code required for creating VR applications. SPECIAL FEATURES OF THIS BOOK This book contains a number of special features that will aid in accomplishing its task of providing inspiration and ideas for its various audiences. One feature to take note of is the VR Discoveries that are included with various applications. When one of those appears, designers are encouraged to take special note because they represent an Aha! by that VR application s designer. Another key feature to this book is the cross reference system described in Chapter 10, and the online database reference on this book s companion website. The cross reference enables developers and interested parties to quickly find different applications in this book that have characteristics that are of interest, regardless of what field the application is in. For example, a reader might be interested to see all the applications that use audio in them. Or, they may be interested in those applications that have been deployed in public venues that use head-mounted displays. A printed book can only present information in a single organization. In this book, applications are grouped according to the area they are used in, such as business, education, or entertainment. Through the use of an online database, the reader can group applications by any characteristic such as the representation scheme, the displays used, or many other criteria. HOW TO USE THIS BOOK We anticipate that many of the readers of this book will skip straight to the chapter that is closest to their area of interest. We encourage that. However, if the reader stops at that point, they will miss some of the most valuable aspects of this book. After that, it would be

14 xiv Preface prudent to read the book cover to cover to discover gems they might not have encountered in their chapter of choice. Of course, the cross references and the online database provide another path to find topics of interest. We believe it will be very beneficial to all readers to read Chapters 1 and 2 to gain an understanding of the field of VR, and also to learn the meanings we are using for a variety of terms. And of course we would be remiss to not encourage any reader who wishes to learn more about virtual reality and its applications to read Understanding Virtual Reality. THE RELATIONSHIP OF THIS BOOK WITH UNDERSTANDING VIRTUAL REALITY As mentioned above, Understanding Virtual Reality is a key resource for anyone who wants a very strong background in virtual reality. It covers VR systems, interaction schemes, and considerable depth in how information can be represented in Virtual Reality applications. The book you are currently reading is where the rubber meets the road, real world VR applications that are foundational to the field. Or the skeptic who wants to see the reality of the field, and understand what is concrete vs. hype will find this book of extreme value. ABOUT THE APPLICATIONS IN THIS BOOK This book provides a glimpse into dozens of virtual reality applications. The applications are from a wide variety of sources, and cover a broad expanse of application areas. The applications in this book are referred to as classic applications in that they are some of the earlier examples of virtual reality put into use. They are each foundational to the field in some way. It is important to note that each of the applications was chosen for a specific reason. We give full coverage of this criteria at the end of Chapter 2, but the litmus test for inclusion are: It must be possible to discuss the application in public. If at all possible, we strove to include applications that we were able to experience ourselves, and/or at least to have direct personal discussions with the application creators. The application illustrates a wide variety of technological implementations. To the extent possible, we include applications that were developed in different development environments, such as Academia, Government, and Private Sector laboratories. We include some applications that have ceased to exist, and some applications that have continued to be used, developed, commercialized, or otherwise exist today.

15 Acknowledgments We wish to thank first and foremost our friends and family, who gave up seeing us as we produced this work. We also thank our home institutions, The National Center for Supercomputing Applications, the Desert Research Institute, and Valparaiso University for their support of our efforts. We would be remiss if we did not acknowledge the fantastic work of all the application developers whose work we describe in this volume and without whom this book would be impossible. Indeed, without them the genre of VR would not exist as we know it. Finally we would like to thank all of our colleagues in the field and our co-workers, from whom we have both learned so much and been so inspired. There were many lynchpins at Elsevier on which the success of this book hinged, but we would like to single out Greg Chalson as our champion. In a sea of turmoil, he stepped forward, took control, and made the publication of this book happen. Thank you. Our other heroes at Elsevier include Mike Morgan and Tim Cox, who started us off, Diane Cerra and Dave Eberly who saw us along, Chuck Glaser who propelled us forward, and Heather Scherer and Paul Gottehrer, the gamesaving closers in the bottom of the ninth. xv