For the purposes of this discussion, we use the terms

Transcription

1 A brief history of stereoscopy R. Duane King We focus on a timeline for stereoscopic visualization. The timeline can be conceived as three distinct eras. The first era was characterized by the full understanding of stereoscopic vision and the development of the stereoscope technology for viewing three dimensional images. The first era was greatly enabled by the simultaneous development of photographic technologies. The second era was characterized by the development of polarized light and anaglyph stereo technology and was mainly manifested with a dramatic outpouring of motion pictures. The third era, the digital era, was developed in connection with virtual reality and scientific data visualization Wiley Periodicals, Inc. How to cite this article: WIREs Comput Stat 2013, 5: doi: /wics.1264 Keywords: stereoscopic visualization; stereoscope; shutter glasses; anaglyph stereo; virtual reality INTRODUCTION For the purposes of this discussion, we use the terms three dimensional (3D) and stereoscopic to mean the same thing; that is, 3D is defined as presenting slightly different images to the left and right eyes so that the human visual system can integrate these two images into a single image that is stereoscopic. Sometimes, the term 3D used to mean an image that obeys the principles of perspective with additional lighting and rendering models so that depth cues can be inferred from the image without a true stereoscopic display. The history of 3D stereoscopic visualization can roughly be divided into three eras. Early stereoscopic visualization featured side-by-side devices coupled with the emergence of photography. Later, stereoscopic visualization enjoyed a popular resurgence based on anaglyph and polarizing filters. Even more recently, a technical resurgence of stereoscopy is based on computerized capabilities coupled with head-mounted displays (HMDs) and shutter glasses. From 1838 to about 1930, devices such as the stereoscope, coupled with newly developing photography, made 3D image viewing a fashionable technology and popular especially among the people of the Victorian era. There is a stereoscope in the Lincoln family home in Springfield, IL 1 (see Figure 1). From about 1950 to perhaps the early 1970s, 3D stereo gained new interest with the increasing use of anaglyph stereo on the printed page and polarized light stereo in the world of motion pictures. Beginning around 1990 the introduction of high refresh-rate cathode ray tube (CRT) projectors and liquid crystal shutter glasses until the current time with 3D high definition television sets, stereo has gained additional interest among scientists as well as movie makers and the general public. The early adoption of shutter glasses and high refresh-rate projectors complemented the growing interest in virtual reality (VR). Correspondence to: king@gmu.edu School of Physics, Astronomy and Computational Sciences, George Mason University, Fairfax, VI, USA Conflict of interest: The author has declared no conflicts of interest for this article. STEREOSCOPIC TECHNOLOGIES Hodges 2 outlined a number of stereoscopic display technologies. More recently Symanzik 3 published an adapted version of Hodges technologies. We present a further, slightly simplified and Wiley Periodicals, Inc. Volume 5, July/August 2013

2 WIREs Computational Statistics History of stereoscopy crystal devices, are the popular choices in the present era of stereo display devices. FIGURE 1 Stereoscope ca (Source: From E. Wegman Collection) abbreviated, adaptation of the Hodges Symanzik taxonomy. Time-Parallel Anaglyph Red green Red cyan Polarized light Separate image Split screen Dual screen Head-mounted Freeviewing Autostereogram Time-multiplex Electro-optical Liquid crystal Shutter glasses Mechanical There are several other techniques which have not gained any substantial following. The stereoscope in the earliest era is an example of a separate image viewer. Anaglyph and polarized light stereo technologies are most represented in the middle era. HMDs and shutter glasses, usually built around liquid PRINCIPLES OF STEREOSCOPIC DISPLAY The underlying principles of stereoscopic (binary) vision were documented by Charles Wheatstone 4,5 ( ). He suggests that Leonardo da Vinci ( ) had some notion that the left and right eyes would see different images, in particular, that an object, for example, a sphere would obscure different background detail for the left eye and the right eye: that a painting, though conducted with the greatest art and finished to the last perfection, both with regard to its contours, its lights, its shadows and its colours, can never show a relievo equal to that of the natural objects, unless these be viewed at a distance and with a singleeye...thetruthofthisobservationistherefore evident, because a painted figure intercepts all the space behind its apparent place, so as to preclude the eyes from the sight of every part of the imaginary ground behind it. 4 Wheatstone 4 concludes: Had Leonardo da Vinci taken, instead of a sphere, a less simple figure for the purpose of his illustration, a cube for instance, he would not only have observed that the object obscured from each eye a different part of the more distant field of view, but the fact would also perhaps have forced itself upon his attention, that the object itself presented a different appearance to each eye. He failed to do this, and no subsequent writer within my knowledge has supplied the omission; the projection of two obviously dissimilar pictures on the two retinæ when a single object is viewed, while the optic axes converge, must therefore be regarded as a new fact in the theory of vision. Wheatstone also discusses Gaspar Monge ( ) and his contributions to descriptive geometry. Wheatstone s 1838 paper, 4 however, fully comprehends stereoscopic vision and describes a device for stereoscopic visualization, albeit large and complicated, and as such must be considered the pioneer in stereoscopic visualization. a In the same general timeframe, there was an early introduction of anaglyph stereo. Symanzik 3 recounts this early history of anaglyph stereo techniques and attributes the earliest work to Rollmann 6,7 and d Almeda. 8 Rollmann 6 introduced side-by-side images colored in red and green with the technique of freeviewing. Later in the same year, Rollmann 7 proposed Volume 5, July/August Wiley Periodicals, Inc. 335

3 wires.wiley.com/compstats using filters corresponding to the same colors for viewing the images. D Almeda 8 also proposed using light of different colors to produce stereoscopic images. It was not until 1901 when Hering 9 proposed a method of using projectors to produce anaglyph stereoscopic images. However, anaglyph stereoscopic technology did not achieve the popular reception that the stereoscope received. THE STEREOSCOPE AND THE BEGINNINGS OF PHOTOGRAPHY A stereoscope is usually a hand-held device for viewing side-by-side 3D stereo images (see Figure 1). Such a device aids the viewer in fusing the images into a standard 3D view. Although the principles of stereoscopic vision were understood, it was not until photography emerged that a viable method of production of stereo images became available. By 1850 a variety of photographic processes including tintypes, albumin, and daguerreotypes were developed. Photographers around the world produced millions of stereoscopic views between 1850 and Their popularity soared when Queen Victoria and Prince Albert received the gift of a stereoscopic viewer at the Crystal Palace exhibition in Soon after, the American jurist Oliver Wendell Holmes called for the establishment of special stereographic collections just as we have professional and other libraries. Around the world, independent and entrepreneurial photographers broke into the growing market for illustrations of all types of subjects: local history and events, grand landscapes, foreign monuments, charming genre scenes, portraits of notables and urban architecture. War and disasters such as floods, fires, train-wrecks, and earthquakes were enormously popular subjects. (New York Public Library 10 ) Oliver Wendell Holmes ( ) together with business man Joseph L. Bates ( ) developed a popular hand-held stereo viewer in Holmes declined to patent his device because he felt that his invention was too simple and obvious. From that time until the 1930s stereoscopic viewing of sideby-side images became a major recreational activity. b See Phillips 11 for a more elaborate discussion of the first era of stereoscopic display devices. GOING OUT TO THE MOVIES The second era began with the popularization of 3D movies. The earliest 3D movies were actually filmed around 1915 using anaglyph glasses, basically as demonstration shorts. Of course, only black and white film would have been available then. Frequently early silent movies would be tinted to set the mood, so some coloring would have been possible. The first known 3D feature movie 12 shown to a paying audience was The Power of Love in September 27, 1922 at the Ambassador Hotel Theater in Los Angeles. This movie was shown by using two film projectors producing an anaglyph image. It is unknown if filters were used on the projectors to produce color or if the films were tinted. This movie is presumed lost. On December 17, 1922, a 3D movie titled The Man from M.A.R.S. was shown at the Selwyn Theater in New York City. This movie was shown with a system called Teleview, c using two interlocked projectors, which alternated showing a left and right eye image. The projectors were linked to handheld viewers the audience used to actuate a shutter alternating the view between the audiences right and left eyes. 13 The Teleview system was a forerunner of the liquid crystal shutter glasses for stereoscopic display. The Teleview system disappeared after the one time run of The Man from M.A.R.S. These were followed by a series of short films, which primarily lasted until the period , when a number of feature length films were made using polarizing filter glasses. d,14 Best known among the first era of 3D movies (that Zone 14 calls the Era of Convergence, ) were: Bwana Devil 1952 Kiss Me Kate 1953 House of Wax 1953 Hondo 1953 Dial M for Murder 1954 Creature from the Black Lagoon 1954 Woods 15 identifies approximately 540 3D films that were theatrically released with some 206 of those released since 2007 and some 88 3D films including short films released between 1952 with Bwana Devil up to The 1980s saw some resurgence in 3D movies although these tended to be B movies, some on the seamier side of movie fare. Of course those films released in the 1950s and through 1980s generally exploited polarized light filters, while releases on Video Home System (VHS) tapes exploited anaglyph technology. The Image Maximization (IMAX) movies were filmed on 65-mm film stock and were drawn through the projector horizontally rather than vertically for conventional film. Early IMAX 3D used polarized stereo glasses and were shot on two separate reels of film stock. This is part of the era that Zone 14 refers Wiley Periodicals, Inc. Volume 5, July/August 2013

4 WIREs Computational Statistics History of stereoscopy to as the Immersive Age, Post-2005, he refers to as Digital 3D Cinema. The film releases in this era generally use shutter glasses. Those released for home use Blu-Ray discs and stereo capable High Definition Television (HDTV) sets. Anaglyph 3D comic books (graphic novels) were also popular in the 1950s concurrent with the 3D movie craze. These were presented in anaglyph form usually with red cyan glasses. Red and cyan are complimentary colors and in an additive color system, e they add to white. In contrast, red and green add to yellow in an additive color system, hence give an overall yellowish cast to the 3D image. 16 Examples of the 1950s era 3D comic books are hard to come by because they are highly collectible and consequently expensive. Applications of anaglyphs to mathematics, statistical methods, and data visualization lagged other disciplines somewhat. Between 1983 and 1986, Daniel Carr and his colleagues described the use of anaglyph stereo in statistical graphics. Symanzik 3 notes also that he and his collaborators independently developed anaglyph stereoscopic statistical graphics between 1992 and Probably an early landmark book containing anaglyph stereo images by Carr, Tom Banchoff, and the late Ruben Gabriel was Wegman and DePriest. 20 More recent work containing anaglyph statistical graphics includes Wegman and Luo 21 and Symanzik. 3 THE COMPUTER TECHNOLOGY ERA: DIGITAL 3D, VR, AND DATA VISUALIZATION The onset of computer technology, even from the 1960s, stimulated interest in digital approaches to visualization. Ivan Sutherland s 1968 HMD 22 was considered as the first VR system. f The HMD began a significant new era in stereoscopic visualization. More modern HMDs are much lighter and more manageable (see Figure 2 for example). Unfortunately, the resolution of HMDs is still relatively poor and not well suited for serious data visualization applications. See Burdea and Coifett 23 for a comprehensive history of VR. However, it was not until the development of more powerful computing environments and the corresponding development of higher resolution display devices exploded the interest in VR. 24 Interest in VR in the early 1990s revived extensive interest in the use of stereo visualization for scientific purposes. The invention of CrystalEyes liquid crystal shutter glasses around 1990 and high refresh-rate CRT projectors allowed for the emergence of highly satisfactory stereo visualization. While ordinary motion pictures have a frame rate of 24 frames per second and 1990s era National Television Systems Committee (NTSC) TV had a frame rate of 30 frames per second, CRT projectors of the 1992 era were capable of displaying 120 frames per second. The CrystalEyes shutter glasses were synchronized using infrared signaling to a computer output at the rate of 60 frames per second for each eye, yielding a highly satisfactory viewing experience. This technology coupled with RISC based, graphicsoriented Silicon Graphics computers provided ideal, if expensive, platforms for experimentation with genuine 3D data visualization frameworks. Cruz-Neira et al. 25 reported on the development of the cave automatic virtual environment (CAVE) (see Figure 3). Cruz-Neira upon graduation took a faculty position at Iowa State University, where she collaborated with Dianne Cook and others to experiment with the development of statistical graphics applications. Dianne Cook and her colleagues reported on their experiences in statistical graphics in the late 1990s with their system named C2. 26,27 This system used Silicon Graphics hardware to display stereo images on three walls (front, left, and right) and the floor. Nelson et al. 26 conducted usability tests on 3D stereoscopic display systems. The authors reported that 3D stereo display of data seemed to allow users to glean more information from the displayed data, but that 3D manipulation of data was harder than using a mouse to manipulate FIGURE 2 A contemporary HMD. Tracking devices allow the computer to determine head orientation and position. These are usually based on magnetic sensors, one of which can be seen over the subject s head. (Source: Photo by E. Wegman) Volume 5, July/August Wiley Periodicals, Inc. 337

5 wires.wiley.com/compstats FIGURE 3 The CAVE environment. (Source: This image is available at produced in the CAVE at the Electronic Visualization Laboratory, University of Illinois, Chicago, IL) the same data in a typical workstation environment (possibly due to a lack of familiarity with 3D input devices coupled with a great familiarity with a mouse). Following the ground-breaking work of Cruz-Neira et al., many installations of CAVE-like environments have been constructed including five- and sixsided CAVEs. Simply Googling CAVE environments will lead the interested reader to descriptions of these environments and comparisons of competing environments. Following the burgeoning interest in CAVE-like environments, a series of International Immersive Projection Technology Workshops 28 were established. Approximately at the same time by 1992 independently of the Cruz-Neira efforts, we installed a similar projection system used for data visualization and statistical graphics also using CrystalEyes shutter glasses, Silicon Graphics computers, and a high refresh-rate CRT projector. The experiences of Wegman and his colleagues with their immersive system were reported in Ref 29. Traditional multiwall CAVE environments require very large spaces and were quite expensive, easily $1,000,000 or more. The one-wall projection system that we had developed was more economical in space (about 400 ft 2 ) and considerably less costly, on the order of $250,000. Multiwall systems that require head tracking are not particularly amenable to multiple simultaneous users, while the one-wall immersive system often needs no head tracking with an approximation to the ideal head position allowing multiple users and stimulating interaction. Other comparisons of virtual environments include those by Brooks, 25 Zielinsky et al., 30 Bowman et al., 31 and Bowman et al. 32 A number of people report discomfort when viewing stereoscopic 3D images. 29 This condition may be caused by the mechanics of the current stereoscopic display systems. When we observe a 3D image in the real world, the muscles in our eyes will converge on the objects in the scene. Another set of muscles will adjust the focus of the scene; this change of focus is termed accommodation. The 3D viewing mechanisms that have been discussed all have a single plane of focus and will always lead to a mismatch in the convergence and accommodation of our optic system. The next generation of 3D viewing systems will be holographic in nature as postulated in Wilkinson 33,34 by Gene Dolgoff. Gene Dolgoff also discusses the mismatch between convergence and accommodation. 33 His display system will essentially create the light interference pattern which would emanate from a 3D scene in a relatively narrow plane of Light Emitting Diode (LEDs) fitted with microlenses. Much as looking at the holographic image on a credit card, our eyes would perceive a more realistic 3D scene and would be able to do this without needing glasses. In the taxonomy we presented in Stereoscopic Technologies, these new technologies would be a variant of freeviewing and autostereogram. Dolgoff also discusses the creation of 3D from 2D images, TV shows, and movies. 34 These technologies seem to be emerging from a number of sources. NOTES a An interesting historical aside is that Charles Wheatstone is often given credit for the invention of the electrical circuit called the Wheatstone Bridge; although it was actually invented by Samuel Hunter Christie in 1833, and later improved by Wheatstone in b It should be noted that broadcast radio became widely accessible in the early 1920s, so that radio tended to supplant stereoscopes as the latest technological marvel. c Another interesting historical aside is that Teleview was invented by Laurens Hammond later known as the inventor of the Hammond organ. d It should be noted that early polarizing filters were linear, one polarized vertically and the other horizontally. This has the unfortunate effect that if the viewer s head tilted, the polarizing effect would be diminished. More modern filters are circularly polarized, clockwise and counterclockwise, minimizing the effect of head tilt. e An additive color system is normally composed of red, green, and blue such as found in color television and movie stock. Green and blue in equal Wiley Periodicals, Inc. Volume 5, July/August 2013

6 WIREs Computational Statistics History of stereoscopy proportions make cyan in an additive system. Thus cyan added to red make white so that red and cyan are complementary colors. Green and red in equal proportions add to yellow, but green and red are not complementary colors. f Sutherland s HMD was nicknamed the Sword of Damocles because it was suspended from the ceiling by a rather imposing apparatus. See Figure 1 in Sutherland. 22 ACKNOWLEDGMENTS The author would like to acknowledge the insightful comments made by the reviewer, which greatly improved the article. The author also wishes to express his thanks to Ed Wegman for the help in revising this manuscript and putting it into final form. REFERENCES 1. National Park Service (NPS 2012). Lincoln Home, National Historic Site, Illinois. Available at: (Accessed May 29, 2012). 2. Hodges LF. Tutorial: time-multiplexed stereoscopic computer graphics. IEEE Comp Graph Appl 1992, 12: Symanzik J. Three-dimensional stereoscopic plots. WIREs Comp Stat 2011, 3: Wheatstone C. Contributions to the physiology of vision. Part I. On some remarkable, and hitherto unobserved, phenomena of binocular vision. Phil Trans R Soc London 1838, 128: Read to the Society June 21, Available at: html. (Accessed April 19, 2012). 5. Wheatstone C. Contributions to the physiology of vision. Part II. On some remarkable, and hitherto unobserved, phenomena of binocular vision, (Continued). Abstracts of the Papers Communicated to the Royal Society of London 1852, 6: Read to the Society January 8, Available at: TC=true. (Accessed April 19, 2012). 6. Rollmann W. Notiz zur Stereoskopie [in German]. Annalen der Physik 1853a, 165: Available at doi: /andp , Rollmann W. Zwei neue stereoskopische Methoden [in German]. Annalen der Physik 1853b, 166: Available at doi: /andp , d Almeida MJC. Nouvel appareil stéréoscopique [in French]. Comptes Rendus des Séances l Academie Sci 1858, XLVII: Hering E. Über die Herstellung stereoskopischer Wandbilder mittels Projectionsapparates [In German]. Pflüger s Archiv 1901, 87: New York Public Library. Stereoscopic photography. Available at: (Accessed April 19, 2012). 11. Phillips D. A brief stereoscopic history. Available at: (Accessed April 19, 2012). 12. IMDb Database. Available at: title/tt /trivia. (Accessed March 12, 2013). 13. Zone R. Stereoscopic Cinema and the Origins of 3-D Film, Lexington, KY: University Press of Kentucky; Zone R. 3-D Revolution: The History of Modern Stereoscopic Cinema. Lexington, KY: University Press of Kentucky; Woods A. The illustrated 3D movie list. Available at: (Accessed April 21, 2012). 16. Wegman E, Said Y. Color theory and design. WIREs Comp Stat 2011, 3: Carr DB, Nicholson WL. EXPLOR4: a program for exploring four-dimensional data. In: Cleveland WS, McGill ME, eds. Dynamic Graphics for Statistics. Belmont, CA: Wadsworth; 1988, Carr DB, Nicholson WL, Littlefield RJ, Hall DL. Interactive color display methods for multivariate data. In: Wegman EJ, Depriest DJ, eds. Statistical Image Processing and Graphics. New York: Marcel Decker; 1986, Carr DB, Littlefield RJ. Color anaglyph stereo scatterplots construction details. In: Computer Science and Statistics: Proceedings of the 15th Symposium on the Interface. New York: North Holland Publishing Company; Wegman E, DePriest D. Statistical Image Processing and Graphics. New York: Marcel Dekker; Wegman E, Luo Q. On methods of computer graphics for visualizing densities. J Comput Graph Statist 2002, 11: Sutherland IE. A head-mounted three-dimensional display. Proceedings of AFIPS 1968, 68: Volume 5, July/August Wiley Periodicals, Inc. 339

7 wires.wiley.com/compstats 23. Burdea GC, Coifett P. Virtual Reality Technology. 2nd ed. Hoboken, NJ: John Wiley & Sons Inc.; Brooks FP. What s real about virtual reality? IEEE Comput Graph Appl 1999, 19: Cruz-Neira C, Sandin DJ, DeFanti TA. Surroundscreen projection-based virtual reality: the design and implementation of the CAVE. SIGGRAPH 93: Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques 1993, doi: / Nelson L, Cook D, Cruz-Niera C. Xgobi vs the C2: results of an experiment comparing data visualization in a 3-D immersive virtual reality environment with a 2-D workstation display. Comput Stat 1999, 14: Cook D, Cruz-Niera C, Kohlmeyer BD, Lechner U, Lewin N, Olsen A, Pierson S, Symanzik J. Exploring environmental data in a highly immersive virtual reality environment.environ Monit Assess 1998, 51: Kunz A. 7th International Workshop on Immersive Projection Technology and 9th Eurographics Workshop on Virtual Environments, Zurich, Switzerland, May, Computer Graphics Forum 2004, 23: doi: /j x. (Accessed March 12, 2013). 29. Wegman E, Symanzik J. Immersive projection technologies for visual data mining. J. Comp. Graph. Statist. 2002, 11: Zielinsky DJ, Brady R, Kim K, Rosenthal Z. Comparison of desktop, head mounted display, and six wall fully immersive systems using a stressful task. Proceedings of IEEE VR 2012 Conference,CostaMesa, CA, 2012, pp Bowman DA, Datey A, Farooq U, Ryu YS, Vasnaik O. Empirical comparisons of virtual environment displays, Technical Report 1, Department of Computer Science, Virginia Tech. 2001, eprints.cs. vt.edu/archive/ /. 32. Bowman DA, Gabbard JL, Hix D. A survey of usability evaluation in virtual environments: classification and comparison of methods. Presence 2002, 11: doi: / Wilkinson S. 3D mayhem, episode 74 of home theater geeks. Available at: home-theater-geeks/74. (Accessed May 29, 2012). 34. Wilkinson S. The future of 3-D, episode 75 of home theater geeks. Available at: show/home-theater-geeks/75. (Accessed May 29, 2012) Wiley Periodicals, Inc. Volume 5, July/August 2013