The perception of 3-D shape from shadows cast onto curved surfaces. J. Farley Norman 1. Young-lim Lee 2. Flip Phillips 3. Hideko F.

Transcription

1 The perception of 3-D shape from shadows cast onto curved surfaces J. Farley Norman 1 Young-lim Lee 2 Flip Phillips 3 Hideko F. Norman 1 L. RaShae Jennings 1 and T. Ryan McBride 1 1 Department of Psychology Western Kentucky University Bowling Green, Kentucky, Department of Psychology Indiana University Bloomington, Indiana, Department of Psychology Skidmore College Saratoga Springs, New York, Running Head: object discrimination from cast shadows Send all Correspondence to: J. Farley Norman Department of Psychology 1906 College Heights Blvd. #21030

2 object discrimination from cast shadows 2 Western Kentucky University Bowling Green, Kentucky, telephone : (270) Farley.Norman@wku.edu

3 object discrimination from cast shadows 3 Abstract In a natural environment, cast shadows abound. Objects cast shadows both upon themselves and upon background surfaces. Previous research on the perception of 3-D shape from cast shadows has only examined the informativeness of shadows cast upon flat background surfaces. In outdoor environments, however, background surfaces often possess significant curvature (large rocks, trees, hills, etc.) and this background curvature distorts the shape of cast shadows. The purpose of this study was to determine the extent to which observers can "discount" the distorting effects of curved background surfaces. In our experiments, observers viewed deforming or static shadows of naturally-shaped objects that were cast upon flat and curved background surfaces. The results showed that the discrimination of 3-D object shape from cast shadows was generally invariant over the distortions produced by hemispherical background surfaces. The observers often had difficulty, however, in identifying the shadows cast onto saddle-shaped background surfaces. The variations in curvature that occur in different directions on saddle-shaped background surfaces cause shadow distortions that lead to difficulties in object recognition and discrimination. PsycINFO classification: 2323

4 Keywords: Form and Shape Perception object discrimination from cast shadows 4

5 object discrimination from cast shadows 5 The perception of 3-D shape from shadows cast onto curved surfaces Shadows are ubiquitous in natural everyday scenes. Shadows may be cast or attached (e.g., see Arnheim, 1974; Knill, Mamassian, & Kersten, 1997). Attached shadows occur on an object when one part of its surface blocks light from reaching other parts. Cast shadows occur when one object blocks light from background surfaces (i.e., the shadow of the object is "cast" onto the background surface). The presence of cast shadows in visual images does not impair the recognizability of natural objects (Braje, Legge, & Kersten, 2000). Indeed, cast shadows have been shown to facilitate the perception of an object's depth, 3-dimensional (3-D) shape, and its movement in space (Kersten, Knill, Mamassian, & Bülthoff, 1996; Norman & Todd, 1994; Norman, Dawson, & Raines, 2000; Norman & Raines, 2002; Wallach & O'Connell, 1953). In addition, painters frequently use shadows to make their paintings appear more "realistic" (e.g., Caravaggio, Claesz, Rembrandt, Wyeth, etc.). Unfortunately, cast shadows have not often been the subjects of psychophysical investigation. In parsing the available literature that does exist regarding the perception and mathematics of cast shadows, it will be helpful to consider the various

6 object discrimination from cast shadows 6 kinds of cast shadows that occur in natural, everyday scenes and the types and quantities of useful information that they contain. When human observers view solid opaque objects, it is necessarily the case that the front parts of those objects occlude, or hide, surface regions belonging to the back parts. The 3-D locus of points on the surfaces of objects that separate visible from invisible regions is a space curve that can be called the occluding bound (Kennedy, 1974), the rim (Koenderink & van Doorn, 1982; Koenderink, 1984a), or the contour generator (Marr, 1982). Researchers refer to the 2-dimensional (2-D) contours that result from the projection of occluding bounds or rims onto the retina by a variety of names: occluding contours, apparent contours, boundary contours, outlines, and profiles. When the direction to an environmental light source is exactly coincident with an observer's line of sight, the cast shadow of an object will have the same shape as its occlusion boundary (e.g., see Plate 19 of Gombrich, 1995). In all other circumstances, however (e.g., when the direction of illumination deviates from an observer's line of sight), the shape of an object's cast shadow will differ from that of the occlusion boundary. For an example of this phenomenon, see Figure 1. In an influential Psychological Review article, Attneave (1954) hypothesized that the most perceptually informative parts of an object's boundary contour were its

7 object discrimination from cast shadows 7 convexities and concavities (i.e., the areas of maximal curvature). Recent results have tended to support Attneave's hypothesis (e.g., Norman, Phillips, & Ross, 2001; De Winter & Wagemans, 2008). In addition, Norman et al. (2000) found that static shadows that contained prominent convexities or concavities were much more recognizable than those that did not. The probable reason that human observers are sensitive to the maximally curved parts of boundary contours and shadows is that these areas contain information about surface shape. For example, Koenderink (1984a) and Richards, Koenderink, and Hoffman (1987) have demonstrated that the convexities and concavities within a 2-D boundary contour correspond to the projections of differently curved regions on the surfaces of 3-D objects. In particular, convex parts of a boundary contour result from the projection of elliptic or "bumplike" areas on the surface of an object, while concave parts result from the projection of hyperbolic, or "saddle-like" surface regions. However, it is important to keep in mind that individual static boundary contours can often be ambiguous or misleading. If a particular projection of an object does not contain significant boundary convexities or concavities, key aspects of the object's "true" 3-D shape will not be revealed (see for example, the right panel of Figure 2). A number of psychophysical studies have evaluated how well human observers

8 object discrimination from cast shadows 8 can identify or recognize objects from static silhouettes and/or cast shadows. The depicted objects range from simple volumetric primitives, such as wedges, cones, and cylinders (Tjan, Braje, Legge, & Kersten, 1995), to assemblages of simple geometric components (Hayward, Tarr, & Corderoy, 1999; Hayward, Wong, & Spehar, 2005), to aircraft (Federico, 1991), to objects encountered in everyday life (animals, manmade objects, body parts, see Wagemans, De Winter, Op de Beeck, Ploeger, Beckers, & Vanroose, 2008), to naturally-shaped objects (bell peppers, see Norman et al., 2000). This past research has shown that objects can be successfully identified from their silhouettes or cast shadows even when their contours change because of object rotation in depth (e.g., see Experiment 3 of Hayward et al., 1999; also Norman, Bartholomew, & Burton, 2008) or because of the movement of environmental light sources (Norman et al., 2000). The addition of motion, either motion of an object or motion of an observer relative to an object, can add information and disambiguate an otherwise ambiguous boundary contour or cast shadow. The motion of an object (e.g., rotation of an object in depth) can serve to "bring" an object's differently curved surface regions to its boundary contour where they can be detected as individual convexities and concavities. In this way, human observers could potentially create a qualitative

9 object discrimination from cast shadows 9 representation of an object's entire shape as an "aspect graph" (see Koenderink, 1984b; Koenderink & van Doorn, 1978). It is possible to go even further than this: for example, Giblin and Weiss (1987), Cipolla and Giblin (2000), Mendonça, Wong, and Cipolla (2001), Wong and Cipolla (2004), and Hernández, Schmitt, and Cipolla (2007) have all demonstrated mathematically that the profiles of moving objects contain a wealth of information about local 3-D surface shape. The work of Cipolla and colleagues is especially noteworthy, because they have shown that it is possible to take a series of actual images, as recorded by a video camera, and reconstruct 3-D models solely from deforming profiles (these methods, however, cannot lead to a completely accurate reconstruction of an object's shape, because surface concavities do not affect the shape of an object's boundary contour). Cortese and Andersen (1991) and Norman et al. (2000) have shown that the presence of motion (i.e., deforming boundary contours or cast shadows) does indeed facilitate human observers' judgments of object shape. The situations and mathematics for many real-world scenes are more complicated than those previously described. In the real world, an observer views solid 3-D objects that are illuminated by one or more environmental light sources. The objects block some of the light emanating from the light sources; as a result cast

10 object discrimination from cast shadows 10 shadows of the objects are projected onto background surfaces. Under these circumstances there is a second special curve on object surfaces that is analogous to, but different from, the rim -- we call it the shadow generator. The shadow generator is the locus of points on the surface of an object (a 3-D space curve) that projects to the shadow on a background surface. The shadow generator is different from the rim, because while the rim is defined relative to the observer's eye, the shadow generator is defined relative to environmental light sources. There will be as many shadow generators as the number of environmental light sources. In the everyday case of an observer looking at a solid object illuminated outdoors on a sunny day, the observer will typically see both the boundary contour (i.e., the projection of the rim) and the shadow cast onto the background surface (i.e., the projection of the shadow generator). The mathematical analyses described earlier (e.g., Cipolla & Giblin, 2000; Giblin & Weiss, 1987; Hernández, Schmitt, & Cipolla, 2007, Mendonça, Wong, & Cipolla, 2001; Wong & Cipolla, 2004) were only designed to recover aspects of 3-D shape from profiles. They were not intended to function in the more general case of cast shadows cast onto arbitrarily curved background surfaces. In a similar vein, nearly all of the limited number of psychophysical studies on this topic have investigated the perception and recognition of objects defined by

11 object discrimination from cast shadows 11 static and moving boundary contours. The main purpose of the current study was to investigate the perception of 3-D object shape from shadows that are cast onto curved background surfaces. In 1992, Knill (see his Figure 8) created a class of stimuli whose contours are frequently perceived by human observers to be shadows cast onto curved background surfaces. It is important to follow up on Knill's observations and carefully examine the effects of curved background surfaces, because natural environmental background surfaces are rarely flat and are often significantly curved. Indeed, Leonardo da Vinci (~1519/1970, see Plate VI, No. 1) pointed out almost 500 years ago that the shapes of cast shadows were as much affected by the shapes of background surfaces as by the shapes of the objects themselves (see Figure 3 for an illustration of this phenomenon, also see Jeaurat, 1750, pp. 203 & 221). The primary goal of the current study was to evaluate the extent to which the human perception of 3-D object shape from shadows is invariant over the distortions that occur when cast shadows are projected onto curved background surfaces. In the current investigation, Experiment 1 was designed to assess how well human observers can perceive and discriminate 3-D object shape when the objects were solely defined by deforming (i.e., moving) cast shadows. The observers were

12 object discrimination from cast shadows 12 initially "trained" with full-cue motion sequences of a set of naturally-shaped objects; they subsequently were required to identify the objects from deforming shadows that were cast onto flat backgrounds. If deforming cast shadows contain perceptually useful information about solid object shape there should be little decrement in performance when the full-cue motion sequences are replaced by deforming shadows. Experiment 2 was designed to extend the results of Experiment 1 by evaluating the perceptual informativeness of object shadows cast onto a variety of curved background surfaces. Experiment 1 Method Apparatus. An Apple imac was used to display the animation sequences. The stimulus patterns were presented on a Mitsubishi Diamond Plus inch monitor. The observers viewed the experimental stimuli from a viewing distance of 100 cm. Stimulus Displays. The objects used were replicas of five ordinary bell peppers (Capsicum annuum, the replicas were made from Smooth-Cast 321 liquid plastic, Smooth-on, Inc.). These five objects, illustrated in Figure 4, are a subset of the natural objects used in our previous investigations (Norman, Norman, Clayton, Lianekhammy, & Zielke, 2004; Norman, Crabtree, Norman, Moncrief, Herrmann, &

13 object discrimination from cast shadows 13 Kapley, 2006; Norman, Clayton, Norman, & Crabtree, 2008). The five peppers were chosen to have similar sizes in order to prevent discrimination based on overall differences in size. Eighty different photographs (full-cue images) or cast shadows (reduced-cue images) of each object were obtained by rotating the objects 360 degrees in depth around a vertical axis in 4.5 degree angular increments. For this experiment, the shadows of the five objects were always cast onto a flat background surface. To generate the shadows, we placed the objects 20 cm in front of the background surface and illuminated the objects and background with a 300 watt light source (a Singer Caramate 3200 projector). The distance between the light source and the objects was 6.5 m. Sample shadows of each of the five objects are presented in Figure 5. A Nikon Coolpix 995 digital camera was used to record the full-cue photographs and the cast shadows at a resolution of 1024 x 768 pixels. In capturing the shadows, the distance from the camera to the objects was 110 cm, while the angle between the camera's line of sight and the direction of the light source was 35 degrees. Once the 800 digital images were acquired (5 objects x 80 frames/object x 2 display types, full-cue photographs vs. cast shadows), they were then transferred to the computer. Procedure. The procedures used were generally identical to those used by

14 object discrimination from cast shadows 14 Norman et al. (2000). All of the observers participated in 5 experimental sessions. Each session consisted of a total of 50 experimental trials (5 objects x 10 trials per object). The order of the object presentations within an experimental block was randomly determined for each block and observer. The observers were required to identify the object (1-5) presented on each trial as depicted by the deforming cast shadows, where the shadow deformation was caused by the object rotation in depth. The individual frames of the apparent motion sequences were updated at a rate of 50 Hz. The observers made their responses by pressing an appropriate key (1-5) on the computer's keyboard. The observers had as much time as they needed on each trial to view the shadows and identify the object. The observers never received feedback regarding their performance during an experimental session. After completion of all 5 experimental sessions, a total of 50 responses had been collected for each of the 5 objects (250 total trials per observer). Prior to the start of an experimental session, all of the observers engaged in a series of practice trials involving full-cue photographs of the objects. These practice trials were used to familiarize the observers with the objects, so that they could perform the object identification task. At the very beginning of the experiment, the observers were unfamiliar with which object was object 1, which was object 2, etc.

15 object discrimination from cast shadows 15 In order for the observers to be able to perform the task, we gave them a series of practice blocks of 10 trials each (2 repetitions x full-cue animations of 5 objects) until their identification accuracy for a practice block reached a criterion of 90 percent or higher correct responses. Feedback, in the form of a short auditory beep, was given to the observers following each correct response in a practice block. Once the 90 percent criterion was reached, the feedback was turned off and the experimental session began. It is important to keep in mind that the observers did not see the deforming cast shadows until the beginning of an experimental block of trials. Because of this, the observers never received any feedback for their performance on trials depicting deforming cast shadows. At the end of the practice trials, the observers could recognize the full-cue depictions at a high level of accuracy (because of the feedback). The key question was whether this high level of performance would transfer to the experimental trials (with no feedback) depicting deforming cast shadows. Observers. The stimulus displays were presented to five observers, three of whom were coauthors (JFN, HFN, & LRJ). Two additional observers (MCW & EM) were naïve with regards to the purposes of the experiment. All observers were either students or faculty at Western Kentucky University. All observers had normal

16 object discrimination from cast shadows 16 or corrected-to-normal visual acuity. Results and Discussion After the very first of the 5 experimental sessions, the observers' identification accuracy was 94, 100, 100, 78, and 96 percent correct for observers EM, JFN, HFN, MCW, and LRJ, respectively. This level of performance, exhibited for the deforming cast shadows, did not differ significantly from the criterion performance of 90 percent correct that was required for the full-cue animation sequences (one-sample t-test, t(4) = 0.9, p =.42, 2-tailed). The observers' total identification performance across all 5 experimental sessions was 98, 100, 100, 95.6, and 99.2 percent correct for observers EM, JFN, HFN, MCW, and LRJ, respectively. It is clear from the current results that human observers can accurately recognize complex solid objects from their deforming cast shadows. All of our observers reported compelling perceptions of solid objects rotating rigidly in depth. The current findings are thus similar to those of our previous research on deforming boundary contours and cast shadows (Norman et al., 2000; Norman & Raines, 2002). It is important to keep in mind, however, that objects with very simple shapes (e.g., single ellipsoids), when depicted solely by their

17 object discrimination from cast shadows 17 deforming boundary contours or cast shadows, can easily appear as non-rigid deformations instead of rigid rotations in depth (e.g., Mach, 1897; Norman & Todd, 1994; Todd, 1985; Wallach & O'Connell, 1953). Our stimuli are consistently perceived as solid objects that rigidly rotate in depth -- the convexities and concavities of the cast shadow boundaries are effective "features" that permit the perceptual recovery of 3-D structure and rigid motion (e.g., see Figure 5). Mach (1897), Wallach and O'Connell (1953), and Todd (1985) have all noted that identifiable features (that can thus be tracked over time) lead to the elimination of perceptual ambiguities and permit the perception of rigid motion in depth. Experiment 2 The results of Experiment 1 demonstrate that our deforming cast shadows (cast onto flat background surfaces) are as informative for the discrimination of 3-D object shape as animations of full-cue photographs. The purpose of Experiment 2 was to determine whether human observers can similarly identify objects when their shadows are cast onto curved background surfaces.

18 object discrimination from cast shadows 18 Method Apparatus. An Apple dual-processor G4 Power Macintosh was used to display the static and deforming cast shadows of the objects. The stimulus patterns were presented on a Mitsubishi Diamond Plus inch monitor. The stimulus displays were accelerated using a Radeon 8500 graphics accelerator card (ATI Technologies, Inc). The observers viewed the cast shadows from a viewing distance of 100 cm. Stimulus Displays. The methods used to generate the cast shadows were the same as those used in Experiment 1. Shadows of the same five objects (Figure 4) were cast onto either flat or curved opaque surfaces. The curved surfaces had a radius of curvature of 14 cm and were either elliptic (a hemisphere) or hyperbolic (a "saddle", or hyperbolic paraboloid) in shape. Figure 6 shows shadows of simple 2-D objects (squares & diamonds) projected onto flat, elliptic, and hyperbolic background surfaces -- note that the straight edges of the objects typically project to curved edges in the resulting shadows cast onto the curved background surfaces (i.e., the distortions are not affine). Sample shadows of object 1 cast onto the flat, elliptic, and hyperbolic background surfaces are shown in Figures 7, 8, and 9, respectively. Elliptic and hyperbolic surface regions were chosen for the curved background surfaces in this experiment, because they are the only generic types of regions that

19 object discrimination from cast shadows 19 exist on the surfaces of arbitrary curved objects (i.e., arbitrary curved object surfaces can be mathematically described or decomposed into elliptic and hyperbolic surface patches -- see Guggenheimer, 1977, pp , and Hilbert & Cohn-Vossen, 1983, pp ). Once again, the shadows were digitally photographed using a Nikon Coolpix 995 digital camera (1024 x 768 pixel resolution). Once captured, the resulting 1200 shadow images (5 objects x 3 background surfaces x 80 shadows/object/background surface) were then transferred to the computer. Procedure. The procedures used were similar to those of Experiment 1. The observers participated in 10 experimental sessions. Each session consisted of a total of 300 trials (30 experimental conditions x 10 trials per condition). The 30 conditions were formed by the orthogonal combination of 5 objects x 3 background surfaces (flat, hemisphere, saddle) x 2 motion types (deforming shadows versus static shadows). The order of the conditions within an experimental block was randomly determined for each block and observer. For a condition depicting a static shadow, one of the 80 captured shadows for an object was chosen randomly. In conditions depicting deforming shadows, full rotations of the objects (i.e., all 80 shadows were presented). The observers' task was to identify the object (1-5) presented on each trial from either type of display (deforming or static shadows). The observers made their

20 object discrimination from cast shadows 20 responses by pressing an appropriate key (1-5) on the computer's keyboard. The observers were given as much time as they needed on each trial to view the shadow(s) and identify the object. The observers never received feedback regarding their performance during an experimental session. After completion of all 10 experimental sessions, a total of 100 responses had been collected for each of the 30 conditions (3000 total trials per observer). Before beginning an experimental block, the observers engaged in a series of practice trials that only involved deforming shadows cast upon the flat background surface. In order for the observers to be able to perform the object identification task, we gave them a series of practice blocks of 10 trials each (2 repetitions x deforming shadows of 5 objects cast upon the flat background surface) until their identification accuracy for a practice block reached a criterion of 90 percent or higher correct responses. Feedback, in the form of a short auditory beep, was given to the observers following each correct response in a practice block. Once the 90 percent criterion was reached, the feedback was turned off and the experimental session began. It is important to note that the observers never saw any static shadows or deforming shadows cast upon the curved background surfaces until the beginning of an experimental block of trials. Because of this, the observers never received any

21 object discrimination from cast shadows 21 feedback for their performance on trials depicting static shadows or deforming shadows cast onto curved background surfaces. When the practice trials were complete, the observers were able to recognize the deforming shadows cast onto the flat background surfaces at a high level of accuracy (because of the feedback). The purpose of this experiment was to determine whether this high level of performance would transfer to the experimental trials (with no feedback) depicting shadows they had never seen before -- i.e., the static shadows and deforming shadows cast upon curved background surfaces. If the observers' good performance for the deforming shadows cast onto the flat background does transfer to the conditions where the shadows are cast onto curved background surfaces, that would represent a significant perceptual accomplishment. Notice from an examination of Figures 5 and 7-9 that the projected shape of the cast shadows is influenced by two independent factors: the particular object (1-5) and the particular background (flat, elliptic, hyperbolic). The observers are required to make a judgment about which object is portrayed on any given trial independent of the background, despite the fact that they have little visible evidence about the nature of the background itself (i.e., the observers can only see the projected shadows, and cannot see the 3-D

22 object discrimination from cast shadows 22 shape of the background). If the observers can accurately discriminate the object shadows when cast upon the curved backgrounds, it would seem reasonable to conclude that they are utilizing shadow boundary features that are invariant with respect to the image distortions created by the various background surfaces. Observers. The stimulus displays were presented to four observers (HFN, RM, MCW, XW) who were either students or faculty at Western Kentucky University. Two of the observers were coauthors (HFN & RM), while the remaining two (MCW & XW) were naive regarding the specific purposes of the experiment, how the stimuli had been generated, etc. Two of the observers (HFN & MCW) had previously participated in Experiment 1 (but of course had never previously seen individual static shadows or those projected onto curved background surfaces). All of the observers had normal or corrected-to-normal visual acuity. Results The observers' results are shown in Figures Figure 10 plots the observers' overall results in terms of d', the measure of perceptual sensitivity used in signal detection theory (e.g., see Macmillan & Creelman, 2005). The observers' d'

23 object discrimination from cast shadows 23 values were subjected to a 2-way within-subjects analysis of variance (ANOVA). The ANOVA revealed that there were significant main effects of both motion (F(1, 3) = 256.3, p <.001, η 2 =.99) and background surface (F(2, 6) = 18.1, p <.005, η 2 =.86). The interaction between motion and background surface was not significant (F(2, 6) = 3.7, p =.09). The observers were able to identify the objects from their deforming shadows at very high levels of accuracy (except for some object shadows projected onto the saddle), but their ability was significantly reduced for static shadows. The overall identification accuracy was 90 percent correct (d' = 2.9) for the deforming shadows and 62 percent correct (d' = 1.4) for the static shadows. The observers' performance was best when the shadows were cast onto the flat and hemispherical background surfaces (83.4 and 79.7 percent correct, respectively), but deteriorated when the object shadows were cast onto the saddle-shaped background surfaces (64.8 percent correct). At this point, it is important to keep in mind that chance performance for this task would be 20 percent correct (d' = 0). Consequently, the observers were able to identify the objects correctly in most instances. Figure 11 plots the observers' results for each of the individual objects. The observers' performance was consistently good when the shadows were cast onto the flat background surfaces. The performances were essentially identical when the

24 object discrimination from cast shadows 24 object shadows were cast onto the hemispherical background surfaces. This invariance in observer performance is impressive, given the fact that the hemispherical background surface creates sizeable non-affine distortions in the shape of the projected object shadows (see Figure 6). Figure 11 also shows that this invariance in performance did not always occur when the object shadows were cast onto the saddle-shaped background surface (a hyperbolic paraboloid). For this background surface, the invariance (i.e., performance similar to the flat plane and hemisphere) only occurred for objects 3-5. Once can see from the figure that there was a reduction in identification performance for objects 1 and 2 when their shadows were cast onto the saddle. This drop in performance was especially severe for the shadows cast onto the saddle by object 1. In order to better understand the observers' detailed responses (for example, why the shadows of object 1 were so confusable when projected onto the saddle), we devised a model -- it operates as follows. The set of planar stimulus projections are treated as a baseline, 'target' set of shadows - one set of 80 views for each of the five objects used in the psychophysical experiment. Elliptic and hyperbolic projections of the same five objects (again, 80 views of each) are treated as the 'test' shadows. The goal of the model is to predict which of the five target objects a given test

25 object discrimination from cast shadows 25 stimulus shadow depicts. To do this, a given test stimulus is compared to the target stimuli by way of the associated boundary contours of each shadow. Each shadow contour is decomposed into various features based upon its local curvature. Negative extrema serve to break the object into parts (e.g., see Hoffman & Richards, 1984; Singh, Seyranian, & Hoffman, 1999), while the positive extrema mark protrusions or "bumps" within parts (these boundary extrema locations agree well with human observers' markings of salient "features", see De Winter & Wagemans, 2008; Norman, Phillips, & Ross, 2001). The boundary of each test shadow (as defined by the locations of the boundary extrema) is then subjected to an affine transformation chosen to minimize the Procrustes distance between the test shadow and a given target shadow (for a description of Procrustes methods in the statistical analysis of shape, see Goodall, 1991). This transformation is repeated for all possible target shadows (80 views encompassing 360 degrees for each of the 5 objects -- a total of 400 fits per test shadow). The minimized Procrustes distances are computed for the 400 possible views and the test shadow is then assigned a target via the minimum average distance from the target. Since there were 800 test shadows (80 orientations of 5 objects cast onto 2 curved background surfaces), a total of 320,000 individual shadow comparisons (test vs. target) were performed.

26 object discrimination from cast shadows 26 Figure 12 plots the performance of the model and the four individual observers for object 1. It is important to keep in mind that the observers were able to successfully identify object 1 when its shadows were cast onto the hemisphere, but their judgments were significantly impaired when its shadows were cast onto the saddle background surface (see Figure 11). It is clear from an inspection of Figure 12 that the model's performance strongly resembled that exhibited by three out of the four human observers (HFN, RM, & XW). It is also clear, however, that observer MCW's performance is unusual, because unlike the other human observers and the model, he was able to identify the shadows of object 1 cast onto the saddle almost as well as those cast onto the hemisphere. When the model was presented with the shadows of object 1 cast onto the saddle, it misidentified the object as being object 4 for 65 out of the 80 possible views/orientations (i.e., the model identified object 1 as being object 4 eighty-one percent of the time). Observer RM's detailed performance was the most similar to that of the model -- he mistakenly identified object 1 as being object 4 forty-three percent of the time. Why does this particular mistake occur? To answer this, consider Figure 13. One can readily see that there is a strong resemblance in shape between the shadows of object 1 cast onto the saddle and those of object 4 cast onto the flat plane (left and middle panels, respectively). Observer

27 object discrimination from cast shadows 27 RM and the model also consistently confused objects 3 and 4. In particular, the model misidentified the shadows of object 3 cast onto the saddle as being object 4 sixty-eight percent of the time, while observer RM exhibited the same confusion 60 percent of the time. Figure 13 also illustrates the strong resemblance in shape between these shadows (object 3 cast onto the saddle vs. object 4 cast onto the flat plane). The errors of observer HFN for object 1 are also understandable given the shadows depicted in Figure 13. When presented with the shadows of object 1 cast onto the saddle, she correctly identified it as object 1 forty-one percent of the time. However, she also mistakenly identified the object 1 shadows as being those of object 3 forty-nine percent of the time. A comparison between the left and right panels of Figure 13 will illustrate that this confusion is not surprising -- depending upon the particular object orientations, the shadows of object 1 cast onto the saddle can strongly resemble those of object 3. It is true that our model exhibits some of the important characteristics of the human judgments. Across all objects, the model performed well when the object shadows were cast onto the hemispherical background surfaces (d' = 1.97) and performed poorly when the object shadows were cast onto the saddle-shaped backgrounds (d' = 0.2). This pattern of good performance for shadows cast onto the

28 object discrimination from cast shadows 28 hemispherical backgrounds and poor performance for shadows cast onto the saddleshaped backgrounds was qualitatively similar to that of the human observers. However, our model cannot explain the totality of the human results. For example, while the model could not correctly identify the object shadows when they were cast onto the saddle-shaped background (d' = 0.2), the human observers fared much better: their average d' value for these static shadows was 1.1. It is clear that while the human observers' performance was reduced when the object shadows were cast onto the saddle-shaped backgrounds (e.g., see Figure 10), they performed much better in those conditions than the model. Discussion The importance of static shadows and silhouettes for human perception has apparently been appreciated for many hundreds of years (Baxandall, 1995; Casati, 2004; Gombrich, 1995), but scientific research on the perceptual informativeness of static shadows and silhouettes is continuing (e.g., De Winter & Wagemans, 2008; Hayward, 1998; Norman, Phillips, & Ross, 2001; Norman, Dawson, & Raines, 2000; Norman & Raines, 2002; Tjan, Braje, Legge, & Kersten, 1995; Tse, 2002; Wagemans, De Winter, Op de Beeck, Ploeger, Beckers, & Vanroose, 2008). The importance of

29 object discrimination from cast shadows 29 motion, however, became clear only in the 20th century. Early research by Miles (1931) and Wallach and O'Connell (1953) demonstrated that the deforming shadows that accompany the rotation of an object in depth lead to the effective perception of both an object's motion and 3-D shape. The early contributions by Miles (1931) and Wallach and O'Connell (1953) were later extended by Todd (1985), Cortese and Andersen (1991), Norman and Todd (1994), Norman, Dawson, and Raines (2000), and Norman and Raines (2002). This recent research, conducted over the past 25 years, has added greatly to our understanding of how deforming shadows and boundary contours contribute to the human perception of 3-D shape and 3-D motion. At the same time, this past research has a number of important limitations. For example, many of these studies utilized objects that possess very simple 3-D shapes (e.g., ellipsoids, see Cortese & Andersen, 1991; Norman & Todd, 1994; Todd, 1985). This is an important limitation, since most naturally-shaped 3-D objects generate relatively complex cast shadows (see for example, Figure 5) whose boundaries contain perceptually important convexities and concavities (Attneave, 1954; De Winter & Wagemans, 2008; Norman, Phillips, & Ross, 2001). The shadows of ellipsoids are unusual in that their outer contours are always convex and never exhibit any concavities (see Figure 1). The shadows of naturally-shaped objects undergoing

30 object discrimination from cast shadows 30 motion also exhibit informative boundary catastrophes (see Koenderink & van Doorn, 1976) that do not occur for the cast shadows of moving ellipsoids. A second limitation that is common to nearly all of the previous studies on this topic is that they investigated the perceptual effectiveness of silhouettes or shadows that were cast onto flat background surfaces. In natural (i.e., non man-made) environments, background surfaces often possess significant curvature. The results of the present experiment show that the human ability to perceive and identify 3-D shape from cast shadows is relatively robust to the distorting effects of curved background surfaces. In fact, for convex hemispherical backgrounds, the observers' performance was essentially identical to that obtained when the shadows were cast onto flat background surfaces. This is an important result for several reasons. First, the distorting effects of the curved background surfaces were large. For example, consider the cast shadows depicted in Figure 6 -- notice that the straight edges of the object project to curved shadow edges on the hemispherical background surface. If a curved edge exists in a cast shadow, it therefore can occur for a variety of reasons: it could derive from a curved object casting a shadow onto a flat background or it could result from an object possessing straight, linear edges (like a polyhedron) casting a shadow onto a curved background. In the general case, then,

31 object discrimination from cast shadows 31 the presence of straight or curved edges in a cast shadow boundary tells us very little about the shape of the original 3-D object, unless we also know the shape of the background surface. The current results are also important in that they suggest future directions for computational modeling. Current computational models (e.g., Cipolla & Giblin, 2000; Giblin & Weiss, 1987; Hernández, Schmitt, & Cipolla, 2007; Mendonça, Wong, & Cipolla, 2001; Wong & Cipolla, 2004) can only recover information about an object's 3-D shape from its silhouette or occlusion boundary contour. They cannot yet recover shape from the more general shadows that are cast onto curved background surfaces. The fact that human observers can perceive and recognize 3-D object shape correctly despite the large distortions in the shape of cast shadows caused by curved background surfaces indicates that computational models can be developed that will function in more general environmental contexts than is currently possible. The current experimental results also showed that the observers' recognition performance decreased significantly when the object shadows were cast onto saddleshaped background surfaces. It is important to note, however, that the overall performance for the saddle conditions was still much higher than chance levels (20 percent), especially when motion was present (77.8 percent correct for deforming

32 object discrimination from cast shadows 32 shadows, 51.7 percent correct for static shadows). The exact reason for the decline in recognition performance for the shadows that were cast onto saddle-shaped backgrounds is not obvious. One possibility is that it is because the shadow deformations themselves are more complicated when shadows are cast onto saddleshaped background surfaces. The additional complexity results from the simple fact that saddle-shaped surfaces curve in depth oppositely in perpendicular directions, whereas hemispherical surfaces possess identical curvatures in all directions. Thus, in the case of saddles, the exact orientation of the principal directions of curvature relative to object parts and features is a critical factor. Consider Figure 14. This figure depicts the shadows of a 2-D object shaped like a "dumbbell" cast onto both hemispherical and saddle-shaped backgrounds. It is readily apparent that the orientation of the object casting the shadow has no effect when the shadows are cast onto the hemispherical surface -- the shapes of the resulting shadows are identical. However, this is not true for the saddle-shaped background surface. In this case, the orientation of object features relative to principal directions of background curvature is important and does affect the shape of the resulting cast shadows. Perhaps this additional complexity in the behavior of shadows accounts for the reduced performance of the observers and the model when they attempted to identify objects

33 object discrimination from cast shadows 33 from shadows cast onto the saddle-shaped background surfaces. In other words, the shapes of shadows cast onto a hemisphere are influenced by three factors (the 3-D shape of the object, the hemispherical shape of the background, and the position of the light source), whereas the shapes of shadows cast onto a saddle are influenced by four factors (the 3-D shape of the object, the saddle-like shape of the background, the exact orientation of the principal directions of background curvature relative to object features, and the position of the light source). In summary, the results of the current experiment have demonstrated that human observers exhibit a considerable degree of robustness in how they perceive 3-D object shape from cast shadows. The observers' performance was relatively unaffected by the large distortions in the shape of the cast shadows that were induced by the presence of curved background surfaces. The ability to discriminate between the various 3-D object shapes was especially good for those shadows that were cast onto convex hemispherical backgrounds.

34 object discrimination from cast shadows 34 References Arnheim, R. (1974). Art and visual perception. Berkeley, CA: University of California Press. Attneave, F. (1954). Some informational aspects of visual perception. Psychological Review, 61, Baxandall, M. (1995). Shadows and enlightenment. New Haven, CT: Yale University Press. Braje, W. L., Legge, G. E., & Kersten, D. (2000). Invariant recognition of natural objects in the presence of shadows. Perception, 29, Casati, R. (2004). Methodological issues in the study of the depiction of cast shadows: A case study in the relationships between art and cognition. The Journal of Aesthetics and Art Criticism, 62, Cipolla, R., & Giblin, P. (2000). Visual motion of curves and surfaces. Cambridge: Cambridge University Press. Cortese, J. M., & Andersen, G. J. (1991). Recovery of 3-D shape from deforming contours. Perception and Psychophysics, 49, De Winter, J., & Wagemans, J. (2008). Perceptual saliency of points along the contour of everyday objects: A large-scale study. Perception & Psychophysics, 70, Federico, P-A. (1991). Measuring recognition performance using computer-based and paper-based methods. Behavior Research Methods, Instruments, & Computers, 23, Giblin, P., & Weiss, R. (1987). Reconstruction of surfaces from profiles. Proceedings of the IEEE First International Conference on Computer Vision,

35 object discrimination from cast shadows 35 Gombrich, E. H. (1995). Shadows: The depiction of cast shadows in western art. London: National Gallery Publications Limited. Goodall, C. (1991). Procrustes methods in the statistical analysis of shape. Journal of the Royal Statistical Society B, 53, Guggenheimer, H. W. (1977). Differential geometry. New York: Dover. Hayward, W. G. (1998). Effects of outline shape in object recognition. Journal of Experimental Psychology: Human Perception and Performance, 24, Hayward, W. G., Tarr, M. J., & Corderoy, A. K. (1999). Recognizing silhouettes and shaded images across depth rotation. Perception, 28, Hayward, W. G., Wong, A. C. -N., & Spehar, B. (2005). When are viewpoint costs greater for silhouettes than for shaded images? Psychonomic Bulletin & Review, 12, Hernández, C., Schmitt, F., & Cipolla, R. (2007). Silhouette coherence for camera calibration under circular motion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29, Hilbert, D., & Cohn-Vossen, S. (1983). Geometry and the imagination. New York: Chelsea. Hoffman, D. D., & Richards, W. A. (1984). Parts of recognition. Cognition, 18, Jeaurat, E-S. (1750). Traité de perspective a l'usage des artistes. Paris: Jombert. Kennedy, J. M. (1974). A psychology of picture perception: Images and information. San Francisco, CA: Jossey-Bass. Kersten, D., Knill, D. C., Mamassian, P., & Bülthoff, I. (1996). Illusory motion from shadows. Nature, 379, 31. Knill, D. C. (1992). Perception of surface contours and surface shape: From

36 object discrimination from cast shadows 36 computation to psychophysics. Journal of the Optical Society of America A, 9, Knill, D. C., Mamassian, P., & Kersten, D. (1997). Geometry of shadows. Journal of the Optical Society of America A, 14, Koendrink, J. J. (1984a). What does the occluding contour tell us about solid shape? Perception, 13, Koenderink, J. J. (1984b). The internal representation of solid shape and visual exploration. In L. Spillmann & B. R. Wooten (Eds.), Sensory Experience, Adaptation, and Perception: Festschrift for Ivo Kohler (pp ). Hillsdale, NJ: Erlbaum. Koenderink, J. J., & van Doorn, A. J. (1976). The singularities of the visual mapping. Biological Cybernetics, 24, Koenderink, J. J., & van Doorn, A. J. (1978). How an ambulant observer can construct a model of the environment from the geometrical structure of the visual inflow. In G. Hauske and E. Butenandt (Eds.), Kybernetik Munich: Oldenbourg. Koenderink, J. J., & van Doorn, A. J. (1982). The shape of smooth objects and the way contours end. Perception, 11, Mach, E. (1897). Contributions to the analysis of the sensations (C. M. Williams, Trans.). Chicago: The Open Court Publishing Company. (Original work published 1886). Macmillan, N. A., & Creelman, C. D. (2005). Detection theory: A user's guide (2nd ed.). Mahwah, NJ: Erlbaum. Marr, D. (1982). Vision: A computational investigation into the human representation and processing of visual information. San Francisco, CA: W. H. Freeman.

37 object discrimination from cast shadows 37 Mendonça, P. R. S., Wong, K-Y. K., & Cipolla, R. (2001). Epipolar geometry from profiles under circular motion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23, Miles, W. R. (1931). Movement interpretations of the silhouette of a revolving fan. American Journal of Psychology, 43, Norman, J. F., Bartholomew, A. N., & Burton, C. L. (2008). Aging preserves the ability to perceive 3-D object shape from static but not deforming boundary contours. Acta Psychologica, 129, Norman, J. F., Clayton, A. M., Norman, H. F., & Crabtree, C. E. (2008). Learning to perceive differences in solid shape through vision and touch. Perception, 37, Norman, J. F., Crabtree, C. E., Norman, H. F., Moncrief, B. K., Herrmann, M., & Kapley, N. (2006). Aging and the visual, haptic, and cross-modal perception of natural object shape. Perception, 35, Norman, J. F., Dawson, T. E., & Raines, S. R. (2000). The perception and recognition of natural object shape from deforming and static shadows. Perception, 29, Norman, J. F., Norman, H. F., Clayton, A. M., Lianekhammy, J., & Zielke, G. (2004). The visual and haptic perception of natural object shape. Perception & Psychophysics, 66, Norman, J. F., Phillips, F., & Ross, H. E. (2001). Information concentration along the boundary contours of naturally shaped solid objects. Perception, 30, Norman, J. F., & Raines, S. R. (2002). The perception and discrimination of local 3-D surface structure from deforming and disparate boundary contours. Perception

38 object discrimination from cast shadows 38 & Psychophysics, 64, Norman, J. F., & Todd, J. T. (1994). Perception of rigid motion in depth from the optical deformations of shadows and occlusion boundaries. Journal of Experimental Psychology: Human Perception and Performance, 20, Richards, W. A., Koenderink, J. J., & Hoffman, D. D. (1987). Inferring threedimensional shapes from two-dimensional silhouettes. Journal of the Optical Society of America A, 4, Singh, M., Seyranian, G. D., & Hoffman, D. D. (1999). Parsing silhouettes: The shortcut rule. Perception & Psychophysics, 61, Tjan, B. S., Braje, W. L., Legge, G. E., & Kersten, D. (1995). Human efficiency for recognizing 3-D objects in luminance noise. Vision Research, 35, Todd, J. T. (1985). Perception of structure from motion: Is projective correspondence of moving elements a necessary condition? Journal of Experimental Psychology: Human Perception and Performance, 11, Tse, P. U. (2002). A contour propagation approach to surface filling-in and volume formation. Psychological Review, 109, Vinci da, L. (~1519/1970). Fourth book on light and shade. In J. P. Richter (Ed.), The Notebooks of Leonardo da Vinci (pp ). New York: Dover. Wagemans, J., De Winter, J., Op de Beeck, H., Ploeger, A., Beckers, T., & Vanroose, P. (2008). Identification of everyday objects on the basis of silhouette and outline versions. Perception, 37, Wallach, H., & O Connell, D. N. (1953). The kinetic depth effect. Journal of Experimental Psychology, 45, Wong, K-Y. K., & Cipolla, R. (2004). Reconstruction of sculpture from its profiles