Recovering 3 D shape: Roles of absolute and relative disparity, retinal size, and viewing distance as studied with reverse-perspective stimuli

Transcription

1 Perception, 2013, volume 42, pages doi: /p7409 Recovering 3 D shape: Roles of absolute and relative disparity, retinal size, and viewing distance as studied with reverse-perspective stimuli Joshua J Dobias 1, Thomas V Papathomas 1,2 1 Laboratory of Vision Research, Center for Cognitive Science, Rutgers University, 152 Frelinghuysen Road, Piscataway, NJ , USA; joshua.dobias@rutgers.edu; 2 Department of Biomedical Engineering, Rutgers University, New Brunswick, NJ, USA; papathom@rci.rutgers.edu Received 1 November 2012, in revised form 15 April 2013 Abstract. When viewing reverspective stimuli, data-driven signals such as disparity, motion parallax, etc, help to recover veridical three-dimensional (3 D) shape. They compete against schema-driven influences such as experience with perspective, foreshortening, and other pictorial cues that favor the perception of an illusory depth inversion. We used three scaled-size versions of a reverspective to study the roles of retinal size, binocular disparity, and viewing distance that influences both vergence and accommodation in recovering the true 3 D shape. Experiment 1 used three conditions, in each of which a parameter was kept fixed across the three stimulus sizes: (a) fixed retinal size, (b) fixed viewing distance, (c) fixed disparity. The predominance of the veridical percept was recorded. Generally, the illusion strength was the same when the viewing distance was fixed, despite significantly different disparities and retinal sizes; conversely, illusion strength changed significantly in fixed-disparity and fixed-retinal-size conditions. Experiment 2 confirmed the results of experiment 1b (roughly equal performances for fixed viewing distance, independent of size) for two additional distances. Viewing distance and scaled disparity (disparity divided by retinal size) are good predictors of the data trends. We propose that disparity scaling is supported by both mathematical and 3 D shape considerations. Keywords: 3 D shape, reverspective stimuli, disparity scaling, vergence 1 Introduction When one views a 3 D object using both eyes, strong monocular and binocular depth cues are produced that depend on many factors, such as the object s 3 D shape, its surface characteristics, its spatial orientation, the position of the light source(s), the viewing distance, etc. Our perception of the object shape is a result of our visual system s interpretation of those cues. Generally, the cues are congruent, thus providing depth and shape information that is consistent across the cues. There are cases, however, that contain incongruent (competing) binocular and monocular depth cues, such as an Ames window, which is a physical trapezoid with strong perspective cues such that, when the short side is closer than the long side, observers misperceive it as a rectangle with a slant that is quite different from the physical slant. A similar form of illusory shape can be perceived when viewing reverspective stimuli (Wade and Hughes 1999; Cook et al 2002, 2008; Papathomas 2002, 2007; Papathomas and Bono 2004; Hayashi et al 2007; Wagner et al 2008; Rogers and Gyani 2010). First developed by artist Patrick Hughes, reverspective stimuli consist of protruding prisms and truncated pyramids with their small faces closer to the observer than their large faces. However, they have realistic scenes painted on their surface containing perspective and foreshortening cues that suggest a 3 D shape opposite to that of the object s actual shape (Wade and Hughes 1999). Depending on viewing conditions, viewers can perceive the real physical depth or the reverse depth, in which far points appear to be closer than near points and vice versa; as a result of the Correspondence may be addressed to either author.

2 Recovering 3-D shape 431 depth inversion, physical concavities are perceived as convexities and vice versa. The main reason why reverspectives are 3 D bistable stimuli is that there are competing influences of visual cues: On one hand, data-driven cues, such as binocular disparity, motion parallax, vergence angle, lens accommodation, etc, favor the veridical physical depth. On the other hand, painted cues, such as perspective and foreshortening, favor depth inversion. Cases like these, where visual cues conflict, allow us to learn how the visual system interprets each type of cue and to better understand each cue s role in perceiving 3 D shape. Parenthetically, similar types of illusory depth inversion can also occur in cases where perspective cues are not deliberately embedded within the stimulus such as Necker cubes, hollow masks (Gregory 1970, 1997; Hill and Bruce 1993, 1994; Papathomas and Bono 2004; Hill and Johnston 2007; Matthews et al 2011) and hollow oval shapes or hollow-potatoes (Hill and Bruce 1994), just to name a few. These stimuli are collectively known as depth-inverting stimuli. Experiments that have studied perceived illusory depth in depth-inverting stimuli have tried to understand how specific stimulus cues such as binocular disparity (Hill and Bruce 1993; Cook et al 2008; Papathomas 2002; Papathomas and Bono 2004; Rogers and Gyani 2010; Matthews et al 2011; Sherman et al 2012), perspective cues (Wagner et al 2008; Sherman et al 2012), motion parallax (Papathomas 2002, 2007; Rogers and Gyani 2010; Sherman et al 2012), and familiarity (Cook et al 2002, 2008; Papathomas 2002, 2007; Papathomas and Bono 2004; Hill and Johnston 2007; Wagner et al 2008; Rogers and Gyani 2010; Matthews et al 2011) influence the perceived shape of the object. Results from these experiments have, as a group, shown that the perception of the 3 D shape depends on whether observers view the stimuli monocularly or binocularly (Hill and Bruce 1993; Cook et al 2008; Papathomas 2002; Papathomas and Bono 2004; Rogers and Gyani 2010; Matthews et al 2011; Sherman et al 2012). For example, when observers start at a large viewing distance that favors the illusion and approach the reverspective stimulus, the illusory percept breaks at a certain flipping distance and gives rise to the veridical percept. This flipping distance is smaller for monocular than binocular viewing, suggesting that binocular disparity plays a role in the illusion. In addition to results with the flipping distance, experiments that assess the predominance of the veridical percept (ie the percentage of time that the object s true 3 D shape is perceived over long observation intervals) also indicate that disparity plays a role because this predominance is stronger for binocular than for monocular viewing (Papathomas and Bono 2004; Sherman et al 2012). However, when one moves closer to a reverspective stimulus, there are several stimulus qualities that change in addition to (horizontal and vertical) binocular disparities, such as the retinal size of the stimulus, the vergence angle, and eye lens accommodation, among others. When approaching or retreating from the stimulus, horizontal disparity magnitudes change inversely with the square of the viewing distance (Mayhew and Longuet-Higgins 1982). A reasonable question is whether and how disparity is scaled to achieve depth constancy (Rogers 2012). Further, an additional reason for isolating the role of binocular disparity is the informal observation that the depth inversion obtained with large reverspectives with widths in excess of 150 cm is as powerful as that of much smaller reverspectives with widths smaller than 30 cm, when both are viewed from comparable distances. At first sight, this appears to be paradoxical, because one would expect the larger binocular disparities of large reverspectives to break down the illusion more easily and thus produce weaker illusions than those of small reverspectives. Of course, the above statement is based on two assumptions that may or may not be valid: (i) the first assumption is that disparity is a dominant cue in determining veridical depth perception; (ii) the second is that there is a monotonic relationship between disparity size and the ability to break the illusion, ie larger disparities favor the veridical percept more strongly than smaller ones.

3 432 J J Dobias, T V Papathomas The experiments described below were designed primarily to study the role of binocular disparity and to test the two assumptions above. Furthermore, we extended the scope to study the roles that retinal size and viewing distance play in recovering the veridical shape of reverspectives. It must be noted that disparity and retinal size are directly available to the visual system as stimulus attributes, whereas viewing distance must be assessed on the basis of available signals. One of the simplest ways to estimate the viewing distance d is to note that the vergence angle v can be obtained from the approximation tan(v) (I/d ) where I is the interpupillary distance. We will use this simple formula to provide a first-order approximation for the vergence angle, even though more elaborate methods exist for assessing the value of d, based on vertical binocular disparities (Mayhew and Longuet-Higgins 1982). (1) It must be noted that lens accommodation also changes as a response to a change in viewing distance. Whether extra-retinal signals, such as vergence and accommodation, are used by the visual system as cues to depth is still debated (Erkelens and Collewijn 1985; Regan et al 1986; Banks and Backus 1998; Wexler et al 2001; Welchman et al 2009). In what follows we will use vergence as the main extra-retinal signal that may provide a cue to the magnitude of the viewing distance, mainly because vergence is simpler to express mathematically than accommodation in terms of the viewing distance. In this paper, we set out to study how binocular disparity is scaled by examining three factors: binocular disparity, retinal size, and viewing distance. To obtain large ranges of values for all these factors, we used three widely different sized reverspective stimuli (small, medium, large) that were scaled versions of each other. Wheatstone (1852) was able to isolate four cues independently (retinal size, binocular disparity, convergence, and accommodation) because he could decouple convergence and accommodation. Convergence is governed by the horizontal disparities while accommodation depends on the viewing distance (between the eyes and the images). By varying disparities independently of viewing distance in his stereoscope, he was able to isolate the effect of the two cues. However, when one uses physical objects, as we did, it is much harder to decouple convergence and accommodation, because they both depend closely on the viewing distance (between the eyes and the point of fixation). Therefore, we were limited to study three cues independently (retinal size, binocular disparity, and viewing distance). Previous work (Hill and Bruce 1993; Papathomas 2002; Papathomas and Bono 2004; Cook et al 2008; Rogers and Gyani 2010; Matthews et al 2011; Sherman et al 2012) has provided evidence that binocular disparity plays a large role in favoring the veridical 3 D percept. Thus, it would be expected that, if the assumption that disparity is a dominant cue is valid, then when the viewing distances are chosen to keep the disparity of the three stimuli fixed, the ability to recover the veridical 3 D shape should be about equal for the three stimuli. Conversely, when disparities vary, then the ability to recover the true 3 D shape for the three stimuli should also vary being stronger for larger disparities. Three conditions were used in the first experiment. In each condition, we kept one variable (retinal size, disparity, or viewing distance) fixed and assessed the ability to recover the true 3 D shape of the stimuli. Contrary to the assumption that binocular disparities are dominant cues, the results of the first experiment indicated that, when the viewing distance was fixed, this ability was roughly the same for the three stimuli, whereas it varied significantly when either the binocular disparity of the retinal size of the stimulus was fixed. The second experiment was designed as a follow up to extend the range of viewing distances to further explore whether the ability to recover the true 3 D shape of the stimuli stays roughly the (1) As proposed by the Mayhew and Longuet-Higgins (1982) algorithm, vertical disparities are used by the visual system to estimate the viewing distance, and then horizontal disparities are used to recover the 3 D structure of the scene.

4 Recovering 3-D shape 433 same for differently sized stimuli that are placed at the same viewing distance. In addition, the results of the second experiment provided valuable data to conjecture how binocular disparities are scaled with distance to achieve 3 D shape constancy. 2 General methods Two experiments were conducted, each of which used the same three stimuli presented at different distances depending on the experimental condition. Stimuli and experimental procedures are described below. 2.1 Participants A total of 75 naive observers (ages years) participated in the two experiments (45 in the first experiment and 30 in the second). Observers were recruited from classes at Rutgers University and received monetary compensation for their time. Each observer reported normal or corrected-to-normal visual acuity, and had normal stereopsis as determined by tests with random-dot stereograms (Julesz 1971). Written consent was obtained from each observer before participating, and the experiments were conducted in compliance with the standards set by the IRB at Rutgers University. 2.2 Stimuli Three 3 D bistable stimuli were used in the two experiments. Reverspective stimuli, first developed by artist Patrick Hughes, consist of truncated pyramids pointed towards the observer with realistic scenes painted on their surface (Wade and Hughes 1999). Experimental stimuli were Kastoria reverspective stimuli (see figure 1) that have been described previously (Wagner et al 2008). Like other reverspective stimuli, the shape of bistable Kastoria stimuli can be perceived in one of two ways, based on either the physical cues from the actual shape (veridical) or from perspective and texture gradient cues painted on the stimulus (illusory) (Papathomas 2002; Papathomas and Bono 2004; Wagner et al 2008). When perceived as illusory, the stimulus appears to be a scene in which two streets recede in the distance, on each side of a convex ( popped out ) building in the center of the image (see figure 1). However, when perceived as veridical, the two truncated pyramids that comprise the stimulus are perceived to protrude toward the viewer and thus the center building appears concave ( caved in ). As described in further detail below, the three experimental stimuli were constructed in three different sizes (small, medium, and large), and were placed at different distances from the observer depending on the experimental condition (fixed retinal size, fixed distance, fixed disparity; see table 1). During experimental trials, stimuli were placed on a stand and were lit from top, bottom, left, and right to eliminate shadows. We used a black curtain as the background that surrounded the entire area of the stimuli to minimize the effect of the shadows cast by the reverspective pieces. Figure 2b is a photograph of the medium-sized stimulus, as it appeared to observers. Further, we designed three different stands, one for each stimulus, to place the center of each stimulus at the same level as the eyes of each observer. Viewing distances were measured from the corner of the central building, which serves as the fixation edge, as shown in figure 2a. In this paper we refer to the horizontal disparity H that is the result of the physical depth difference z (along the line of sight) between the fixation edge and the point on the reverspective nearest to the observer (see the top view in figure 1). We designed the three stimuli to form a geometric sequence, using a linear-dimension factor of approximately The dimensions of the small stimulus were 30.2 cm wide, 18.1 cm tall, and 5.03 cm deep; hence the widths of the medium and large stimuli were 71.3 and cm, respectively, with the heights and depths varying proportionally.

5 434 J J Dobias, T V Papathomas Top view toward viewer Front view Side view toward viewer Painted front view Figure 1. [In color online, see Kastoria reverspective stimulus used in the experiments. Orthographic front, top, and side views (top panel), and a photograph of the stimulus front (bottom panel). w z d (a) I (b) Figure 2. [In color online.] (a) Basic stimulus setup for experiments 1 and 2 (not drawn to scale). (b) A photograph of the experimental setup with the medium-sized stimulus.

6 Recovering 3-D shape Procedure Each observer sat in a chair facing one of the reverspective stimuli and placed their chin on a chin-rest to maintain head location. Observers were asked to keep their eyes focused on a location in the center of the stimulus and to move their eyes as little as possible. The exact maintenance of fixation was not monitored. Experimental sessions consisted of six 2 min trials in which the observer viewed each of the reverspective stimuli twice and pressed one of two keys on a keyboard to indicate how he/she perceived the stimulus to be shaped. If the center building appeared to be popped out or caved in, observers were asked to press and hold the left or right arrow key, respectively. Observers viewed each stimulus (small, medium, or large) twice in an experimental session and the order of presentation was randomized for each observer. For all experiments, Mathematica was used to record the duration of each occurrence of pressing the two buttons and to compute the percentage of time (predominance) that each observer perceived as popped out or caved in. The predominance of the veridical percept, ie the percentage of time that the observer spent perceiving the veridical shape, was recorded and used to assess the strength of each stimulus cue. Predominance has been successfully used as a measure of the strength of the veridical percept (Papathomas and Bono 2004; Sherman et al 2012). Data for each observer, then, are the average of the two predominance times for each of the 2 min trials that the observer viewed each size stimulus. To begin each trial, a sound (short beep) was played to indicate to the observer when to start pressing the left or right button. After the 2 min trial was complete, a second sound indicated the end of the trial. 3 Experiment 1 To better understand the roles that retinal size, horizontal disparity, and viewing distance play in recovering the veridical caved in shape, experiment 1 employed three separate conditions. Condition 1a: In the first fixed retinal size condition, the three stimuli (small, medium, and large) were each placed at a specific distance from the observer to maintain a common retinal size for each stimulus; of course, this produced different disparities and different fixational vergence angles for the three stimuli; specifically, since retinal size varies inversely with the viewing distance d, whereas disparity varies inversely with d 2, as we select the distances to keep the retinal sizes fixed, the disparity decreases as the stimulus size increases (see table 1). Condition 1b: In the second fixed distance condition, each of the three sized stimuli was placed at a common viewing distance; each stimulus, therefore, had a different disparity and retinal angle (retinal size), but had a common vergence angle and accommodation for the physical fixation point. It is also important to note, however, that despite having a common vergence angle for the physical fixation point, there may have Table 1. Viewing distances and horizontal disparities for fixed retinal size, fixed distance, and fixed disparity in experiment 1. Small Medium Large Viewing distance/cm Fixed retinal size Fixed distance Fixed disparity Estimated horizontal disparity/min Fixed retinal size Fixed distance Fixed disparity

7 436 J J Dobias, T V Papathomas been separate vergence angles for the perceived fixation point when perceiving the illusory shape (Wagner et al 2008). Condition 1c: Finally, in the third fixed disparity condition, each stimulus was placed at a distance that produced a common binocular disparity but different retinal sizes, as well as different vergence angles, and lens accommodation values. The viewing distance for the medium-size stimulus remained constant in all three conditions, and the viewing distances for the small and large stimuli were adjusted based on the distance of the medium stimulus. For each condition, we employed 15 naive observers (45 total). The estimated horizontal disparities for all three conditions can be seen in table 1. The horizontal disparity H is estimated using the approximation H Iz/d 2 (Mayhew and Longuet-Higgins 1982) where I is the interpupillary distance (for simplicity in obtaining viewing distances, we use a fixed value for I = 6.5 cm), z is the physical depth (see figure 2a), and d is the viewing distance. Actual disparities varied slightly for each observer due to individual differences in interpupillary distance. As seen in table 1, the disparity pattern reverses between the fixed retinal size (disparities decrease with increasing size) and the fixed distance conditions (disparities increase with increasing size). Also, for the medium sized stimulus, disparity remained the same for each condition. The distances that were chosen for each condition can also be seen in table 1. Specific details concerning how each distance was chosen for the three conditions are provided in the Appendix. 3.1 Results The predominance times, averaged across observers, for the veridical percept of each stimulus in each condition are shown in figures 3a, 3b, and 3c for experimental conditions 1a, 1b, and 1c, respectively. An overall mixed-factor ANOVA was not conducted on the data because the stimuli (small, medium, large) in each condition were placed at different distances. Instead, an individual repeated-measures ANOVA was conducted for data in each of the three conditions (fixed retinal size, fixed distance, and fixed disparity). Finally, t tests were conducted to determine differences between the average predominance values for the veridical percept that was produced by each stimulus Veridical shape/% small medium large small medium large small medium large Stimulus Stimulus Stimulus (a) (b) (c) Figure 3. Average proportion of time observers spent perceiving the veridical shape of the reverspective stimulus for the small (light gray), medium (white), and large (dark gray) stimulus in the fixed retinal size (a), fixed distance (b), and fixed disparity (c) conditions in experiment 1. Error bars represent ±1 SEM. Disparity values for each stimulus/condition (min) are shown above each bar.

8 Recovering 3-D shape 437 The individual repeated-measures ANOVAs showed significant differences among average veridical predominance values produced by stimuli in the fixed retinal size condition (F 2, 28 = 27.54, p < 0.05, η 2 = 0.663) and in the fixed disparity condition (F 2, 28 = 9.11, p < 0.05, η 2 = 0.394), but not for the fixed distance condition (F 2, 28 = 0.915, p > 0.05, η 2 = 0.061). Planned t tests showed that, in the fixed retinal size condition, the small stimulus produced higher predominance values than the medium stimulus (t 14 = 4.1, p < 0.05) and the large stimulus (t 14 = 7.42, p < 0.05); also, the medium stimulus produced higher predominance than the large stimulus (t 14 = 3.38, p < 0.05). In the fixed disparity condition, t tests showed that the small stimulus produced a higher predominance time than the medium stimulus (t 14 = 2.19, p < 0.05) and the large stimulus (t 14 = 4.34, p < 0.05). 3.2 Discussion and further analyses The dual assumption that (i) disparity is a dominant cue in recovering the true depth and (ii) that increasing horizontal disparities would increase the ability to recover the true 3 D shape of the stimulus appears to be confirmed by the data in the fixed retinal size of figure 3a. Indeed, predominance decreases with increasing stimulus size, just as disparity decreases with increasing size when the retinal size is held fixed. However, in the fixed distance condition, the magnitudes of horizontal disparities increase as stimulus size increases, whereas predominance values remained constant across stimuli. Further, in the fixed disparity condition, where one would expect equal predominance values under the dual assumption, there were significant differences between the predominance of small stimuli as compared to large stimuli, despite having equal binocular disparities. Overall, it is clear from the data that disparity may not necessarily play a large role in recovering the veridical 3 D shape of the reverspective stimuli used in this experiment. Similarly, the data indicate that retinal size does not appear to be a determining factor, either. The data do, however, confirm findings from earlier studies that viewing distance plays a major role; our data reveal that its role is much larger than that of disparity or retinal size. To clarify the influence of viewing distance, data were reorganized into three groups based on stimulus size (small, medium, and large) and distance. Individual between-groups ANOVAs were conducted for data in each of the three reorganized size groups (small, medium, and large). Individual ANOVAs showed significant differences between average predominance times produced by different distances of the small stimulus (F 2, 42 = 4.43, p < 0.05, η 2 = 0.174) and the large stimulus (F 2, 42 = 7.35, p < 0.05, η 2 = 0.259). As expected, it did not show significant differences for the medium stimulus, as it was always placed at a constant distance from the observer (F 2, 42 = 0.23, p > 0.05, η 2 = 0.011). To summarize, the first individual repeated-measures analysis showed that viewing distance might play a larger role than horizontal disparity in determining how an observer perceives the shape of a reverspective stimulus. The most remarkable finding in experiment 1 was that the veridical predominance for a fixed viewing distance was approximately the same for all three stimuli, despite the extensive differences in the sizes of the stimuli that caused significant differences in retinal size and in disparity. This motivated us to conduct experiment 2, in which we placed the three stimuli at the same viewing distance, using two additional viewing distances, to test the hypothesis that the ability to recover the 3 D shape depends strongly on the viewing distance. The additional advantage for conducting experiment 2 was that it provided valuable data on how disparity is scaled with distance. 4 Experiment 2 To test whether veridical predominance is roughly the same if differently sized stimuli are viewed from the same viewing distance, experiment 2 added two more fixed distance conditions. Condition 2a ( near distance ): Set at half the original fixed distance condition

9 438 J J Dobias, T V Papathomas (267.5 cm) at cm. Condition 2b ( far distance ): Set at twice that distance, namely 535 cm. We employed 15 naive observers for each condition for a total of 30; there was no need to repeat the original condition at cm. In experiment 2, there are large differences in disparity and retinal sizes among the stimuli, as in experiment 1 (see table 2). Despite these differences in disparity and retinal size for the small, medium, and large stimuli, results from experiment 1 predict equal but higher predominance times in the near distance ( cm) condition and equal but lower predominance times in the far distance (535 cm) conditions. Table 2. Horizontal disparities for near and far conditions in experiment 2. Horizontal disparity/min Small Medium Large Near Far Results As in experiment 1, each observer viewed the three experimental stimuli twice and the predominance times for each stimulus were averaged to give one value per stimulus for each observer. The average predominance times for each stimulus in each of the conditions can be seen in figure 4. An overall mixed-factor ANOVA was conducted showing that there was a main effect of viewing distance on predominance (F 2, 42 = 21.01, p < 0.05, η 2 = 0.50), but that there was no main effect of stimulus size (F 2, 84 = 0.653, p > 0.05, η 2 = 0.015). Also, there was no interaction between distance and stimulus size (F 4, 84 = 0.615, p > 0.05, η 2 = 0.028). Since the overall ANOVA showed no differences of stimulus size (small, medium, large), data for each observer were collapsed across the three stimuli in each condition and t tests were conducted to determine differences among distance conditions (133.75, 267.5, and 535 cm). Comparisons showed that stimuli at cm had higher veridical predominance times than stimuli at cm (t 28 = 3.515, p < 0.05) and at 535 cm (t 28 = 6.023, p < 0.05). Also, stimuli at cm had higher predominance times than stimuli at 535 cm (t 28 = 3.204, p < 0.05). Veridical shape/% Stimulus size small medium large Distance/cm Figure 4. Average proportion of time that observers spent perceiving the veridical shape of the reverspective stimulus for the small (light gray), medium (white), and large (dark gray) stimulus at distances of cm (left) cm (middle, from experiment 1), and 535 cm (right) in experiment 2. Error bars represent ±1 SEM. Disparity values (min) for each stimulus/condition are shown above each bar.

10 Recovering 3-D shape Discussion and further analysis Remarkably, the results of experiment 2 demonstrated that the finding of non-significant differences in veridical predominances for the fixed-distance condition of experiment 1b was extended to a wide range of viewing distances, from half to twice the original fixed viewing distance of experiment 1b. Namely, despite large differences in disparity and retinal size for each of the three stimuli, when stimuli were presented at the same viewing distance (same vergence angle and accommodation), the portion of time spent perceiving the veridical stimulus shape was not significantly different for the three stimuli. Further, as expected, veridical predominance decreased as the viewing distance increased. Experiment 2 also provided us with additional data that we use below to explore possible ways of scaling binocular disparities to recover the true 3 D shape of objects, as discussed below. 5 General discussion The present experiments were designed to study the role of three cues in recovering 3 D shape: binocular disparity, retinal size, and viewing distance. Regarding disparity, we tested two hypotheses: (i) that disparity is a dominant cue for recovering true depth; (ii) that the ability to recover true depth improves with increasing disparity magnitude. With respect to the first assumption, there is converging evidence that the role of binocular disparity depends strongly on the nature of the stimuli viewed by the observer, with its role ranging between the two opposite extremes of negligible and predominant. Specifically, Erkelens (2012a, 2012b) observed that disparity-derived depth estimates can play a significant role in some cases (eg random-dot stereograms; Julesz 1971), a moderate role in other cases (eg hollow-mask illusion; Hill and Bruce 1993; Hill and Johnston 2007), and a weak-to-nonexistent role in others (bistability of luminance-defined Necker cubes that contained disparity cues; Erkelens 2012a). This motivated Erkelens (2012a, 2012b) to propose a framework for binocular vision, according to which the depth estimate derived from binocular disparity is not necessarily combined with the depth estimates that are derived from other (mostly monocular) depth cues using appropriate weights. Instead, the disparity-based depth estimate may or may not be combined with the other cues to depth. Pizlo and Steinman (2008), for example, speculate that disparity plays a role only when the other depth cues provide weak depth estimates. Our results, indicating a moderate role of disparity, provide supporting evidence in favor of the proposals made by Erkelens (2012a, 2012b) and Pizlo and Steinman (2008). With respect to the second assumption about the monotonic relationship between disparity magnitude and the ability to recover true depth, the results of our experiments, when each stimulus size is considered separately, are in agreement with this assumption. However, the range of disparity values was not adequately large to test this monotonic relationship, which may not hold for excessive disparity values. The largest disparity we employed across all conditions was only 34.9 min of arc, which specifies a reasonably limited range of disparities that did not test monotonicity for extreme values. The results of experiment 1 indicated that equal retinal sizes and equal disparities for the three stimuli did not produce equal predominances for the veridical percept, whereas equal viewing distances did. These results are somewhat unexpected for at least three reasons. First, results from previous work (Hill and Bruce 1993, 1994; Cook et al 2002, 2008; Papathomas 2002, 2007; Rogers and Gyani 2010; Matthews et al 2011; Sherman et al 2012) suggest that the veridical predominance should be different when stimuli of different size are placed at equal distances because the disparities are unequal. Second, if properly fixated throughout each trial, vergence angle and accommodation alone only indicate the distance to fixation and not the actual shape, which requires differences in their values and hence multiple binocular fixations. Third, vergence angle becomes a weaker indicator of distances beyond 2 m (see Howard and Rogers 2002, p. 402).

11 440 J J Dobias, T V Papathomas To further understand and to extend the findings from experiment 1, a second experiment was conducted to test predominance for the three stimuli at a shorter (by a factor of 0.5) and a longer (by a factor of 2.0) distance from the observer than the fixed distance condition in experiment 1. When the distance was decreased, the disparities for the three stimuli grew larger and more disparate across stimuli, whereas, when the distance was increased, the disparities grew smaller and more similar. Results confirmed the expectations suggested by the pattern in experiment 1. The veridical predominance values were not significantly different across stimulus size when these stimuli were placed at the same viewing distance for all three magnitudes of viewing distance. Together, data from experiments 1 and 2 can be used to explore the dependence of the predominance of the veridical percept on retinal size, binocular disparity, and the combination of accommodation and vergence angle at fixation that are produced by the stimuli, as shown in figures 5a, 5b, and 5c, respectively. These figures show these dependences by plotting all 15 data points obtained in both experiments, using different symbols for the small, medium, and large stimuli Veridical shape/% Retinal angle/deg Disparity/min Distance/cm (a) (b) (c) Figure 5. Average proportion of time spent perceiving veridical shape as a function of (a) retinal angle, (b) disparity, and (c) viewing distance for the small (gray, solid), medium (gray, dashed), and large (black, dashed) sized stimuli for both experiments 1 and 2. Solid black line represents the overall fit to all data points. Error bars represent ±1 SEM. We next examine the role of each of the components. (i) As seen in figure 5a, there is a relationship between predominance and retinal size for each individual stimulus size: for small size (b = 2.80, t 3 = 7.13, r 2 = 0.944, F 1, 3 = 50.81, p < 0.05); for medium size (b = 1.47, t 3 = 12.49, r 2 = 0.981, F 1, 3 = , p < 0.05); for large size (b = 0.87, t 3 = 8.9, r 2 = 0.963, F 1, 3 = 79.15, p < 0.05). A quick glance at figure 5a indicates that there is no global correlation for all 15 points and retinal size; indeed, retinal size does not predict predominance (b = 0.39, t 13 = 1.52, r 2 = 0.151, F 1, 13 = 2.32, p > 0.05). (ii) Similarly, the relationship between predominance and horizontal disparity (figure 5b) is strong for each individual stimulus size: for small size (b = 3.79, t 3 = 4.57, r 2 = 0.874, F 1, 3 = 20.89, p < 0.05); for medium size (b = 2.18, t 3 = 9.12, r 2 = 0.965, F 1, 3 = 83.24, p < 0.05); for large size (b = 1.17, t 3 = 4.63, r 2 = 0.877, F 1, 3 = 21.47, p < 0.05). The global relationship between veridical percept and disparity is significant (b = 1.08, t 13 = 2.87, r 2 = 0.387, F 1, 13 = 8.22, p < 0.05), but the fit is poor compared to viewing distance, as detailed below. (iii) The relationship between veridical percept and viewing distance is significant (figure 5c) for small size (b = 0.076, t 3 = 5.27,

12 Recovering 3-D shape 441 r 2 = 0.902, F 1, 3 = 27.81, p < 0.05) and for the large size (b = 0.087, t 3 = 7.94, r 2 = 0.954, F 1, 3 = 62.98, p < 0.05), but falls short for the medium size (b = 0.069, t 3 = 2.75, r 2 = 0.716, F 1, 3 = 7.56, p = 0.07). However, viewing distance does provide a much better global fit to the observed predominance times for all 15 data points regardless of stimulus size (b = 0.081, t 13 = 10.10, r 2 = 0.886, F 1, 13 = , p < 0.05). Varying viewing distance produces changes to vergence angle and to lens accommodation. We examine these two factors next. Whether the oculomotor signals that control vergence eye movements are used as a cue to depth is a long-standing debate (see, for example, Viguier et al 2001). Vergence angle only indicates the distance to fixation and not the actual shape of the stimulus. If vergence angle were actually used by the visual system (Banks and Backus 1998; Wexler et al 2001; Welchman et al 2009), one would need to use differences in vergence angle between the fixation edge and other features on the stimulus to recover the actual shape. Observers may have directed their fixation to points around the center of the stimulus, even though they were instructed to stay properly fixated at the stimulus center throughout each trial; we did not monitor eye movements in our experiments. Also, because vergence angle becomes a weak indicator of depth at distances greater than 2 m (Howard and Rogers 2002, p. 402), it is unlikely that the visual system makes use of vergence angle alone to perceive the veridical shape of the stimulus for long viewing distances in our experiments. Accommodation, the second factor that can potentially provide cues for the magnitude of the viewing distance, can be influenced by apparent depth when viewing both 2-D and 3 D stimuli that contain pictorial cues (Takeda et al 1999; Busby and Ciuffreda 2005). There are two possible ways that accommodation can contribute to recovering the true depth of a scene: (i) The oculomotor signals for accommodation may be used to assess viewing distance (Morrison and Whiteside 1984; Judge and Cumming 1986). To our knowledge, there is no convincing evidence that these oculomotor signals provide strong cues for estimating the viewing distance. (ii) The blur gradient is another possible cue, because only the fixation point and its immediate neighborhood are in sharp focus by virtue of proper accommodation; points away from fixation cause different levels of blur that can be used to recover depth. Much of the research on blur s role in determining depth has been conducted in relation to disparity and has been tested using 3 D display monitors rather than real stimuli (Watt et al 2005; Hoffman et al 2008; Held et al 2012). Blur gradients have been shown to be a useful indicator of depth within a scene. In addition to the widely accepted evidence that blur provides a qualitative cue to depth (Mon-Williams and Tresilian 2000; Watt et al 2005; Hoffman et al 2008), recent studies have proposed that blur contributes a quantitative cue to depth (Held et al 2012; Vishwanath and Blaser 2010). It is possible that participants in our experiments used the differential blur gradient (more accelerated for parts of the scene that are closer to the fixation point than parts beyond fixation) as a cue to depth. It is obvious from figure 5b that disparities must be scaled to improve their ability to recover the true 3 D shape of objects. This scaling can be approached on purely mathematical grounds. Since horizontal disparities vary inversely with d 2, the square of the viewing distance, it is sensible to scale them by the angular size, which varies inversely with d. The scaled disparities will vary inversely with d, and since the global correlation of predominance and d is strong (figure 5c), we expect the correlation of predominance and scaled disparities to be strong. This is indeed the case, as shown in figure 6. In fact, this relationship is very strong (r = 0.965, p < 0.05) and the scaled disparity strongly predicts the veridical predominance (b = 87.74, t 13 = 13.27, r 2 = 0.93, F 1, 13 = , p < 0.05) for all data points regardless of stimulus size. Beyond the mathematical reasoning, scaling the disparities by the angular size of an object also makes sense from a 3 D shape recovery viewpoint. Since the height, width, and depth of an object provide good measures for its overall 3 D shape, we can assess

13 442 J J Dobias, T V Papathomas the 3 D shape by pairwise ratios of these dimensions: height-to-width and depth-to-width (the remaining ratio of depth-to-height is determined by the other two). To achieve 3 D shape constancy, the perceived magnitudes of these ratios must not vary as a function of viewing distance (Richards 2009). A first-order approximation for the aspect ratio of width-to-height can be obtained directly from a monocular retinal image, without the need for disparity. However, to get the ratio of depth-to-width, one has to scale disparity (that provides an estimate of depth) by retinal size (that provides an estimate of width). This is the rationale on why scaling disparity by retinal size is expected to provide a good fit to the data, as indeed is the case (figure 6) Veridical shape/% Disparity/Retinal angle Figure 6. Average proportion of time spent perceiving veridical shape as a function of scaled disparity (disparity divided by retinal angle) for the small (gray, solid), medium (gray, dashed), and large (black, dashed) sized stimuli for both experiments 1 and 2. Solid black line represents fit to all data points. Error bars represent ±1 SEM. Previous work (Prince and Rogers 1998) has shown that peak-to-trough disparity thresholds depend on the corrugation frequency, with larger eccentricities producing higher thresholds (Prince and Rogers 1998, figure 3). We computed the fundamental spatial frequencies of the depth undulations in our stimuli and we obtained the corresponding eccentricities. We verified that our stimuli provided disparities that were above the thresholds reported by Prince and Rogers (1998, figure 2), thus providing usable supra-threshold disparity signals for recovering 3 D shape. The horizontal and vertical size ratios (HSR and VSR) are two cues that could be used in the current experiments (eg Backus et al 1999). They are relevant but they have the limitation of being applicable to an isolated surface rotated about a vertical axis, whereas our stimuli comprise multiple planar surfaces. Because HSR by itself is ambiguous, one needs it in combination with the VSR, which is itself independent of surface slant but varies with the position of the surface patch with respect to the head (Backus et al 1999). Indeed, the surfaces in our stimuli occupy different positions in space and they can provide useful HSR and VSR signals for recovering the true slant angles. An additional cue that may contribute to recovering the true 3 D depth, pointed out by Brian Rogers, is the changing retinal aspect ratio of two edges that are located at different depths. Consider, for example, the retinal ratio r = M'/m' of the central distant long vertical edge M and the short near vertical edge m of the building nearest to viewers (M' is the retinal projection of M; ditto for m' and m). For very large viewing distances, M'/m' approaches the true physical ratio M/m in the real-world stimulus. As the viewing distance decreases, M'/m' decreases; theoretically, the value of M'/m' can be used to obtain an estimate of the viewing distance, if one starts at large viewing distances that provide an estimate of the true ratio M/m. We did not pursue additional experiment to test whether viewers used this cue in our paradigm.

14 Recovering 3-D shape 443 Reverspective stimuli contain strong conflicting monocular and binocular cues to the shape of the stimulus. As observers approach a reverspective stimulus, the disparity increases, possibly causing a perceptual break from the illusory percept. If vergence is used as a cue to depth by the visual system, then differences in vergence angles between various points on the object, fixated in sequence, also increase as distance decreases, also possibly contributing to a perceptual switch. Rogers and Gyani (2010) state that, when approaching a reverspective stimulus, the binocular disparities eventually win out and the observer perceives the veridical shape (Rogers and Gyani 2010, p. 332). It is possible (and quite likely) that when the illusory percept breaks as an observer approaches a reverspective stimulus, cues that may provide signals for recovering the true depth (such as binocular disparity, motion parallax, vergence, accommodation, etc) win out and they cause the perceptual switch. It is important to note that the distances at which the stimuli were presented in the experiments described above are greater than the typical distances (below 1 m) at which the illusory percept typically breaks down (Cook et al 2002, 2008; Papathomas 2002, 2007; Rogers and Gyani 2010). For the distances used in our experiments, then, the illusory percept is a result of the perspective and other monocular cues winning out over the cues that signal the true depth. In summary, we obtained evidence for a reduced role of binocular disparity and an important role of viewing distance in recovering the true 3 D shape of physical reverseperspective stimuli. Previous findings have established that the illusion strength depends on viewing distance, but all the studies used stimuli of fixed sizes (Cook et al 2002; Papathomas 2002; Papathomas and Bono 2004; Rogers and Gyani 2010). The novel finding of our experiments is that the strength of the illusion varies with viewing distance independently of stimulus size. It appears that, when the visual system receives strong conflicting signals for recovering 3 D shape, such as binocular disparity and linear-perspective cues, it relies on other cues that it uses all the time to determine the true 3 D shape of objects. Two such potential cues, which are not mutually exclusive, may contribute to disambiguating the conflict. One is the vergence angle, possibly combined with cues from accommodation and the resulting blur. The other potential cue is the binocular disparity scaled by the retinal size of the object; this scaling is sensible based on both mathematical (distance scaling) and 3 D shape (aspect ratio) considerations. References Backus B T, Banks M S, van Ee R, Crowell J A, 1999 Horizontal and vertical disparity, eye position, and stereoscopic slant perception Vision Research Banks M S, Backus B T, 1998 Extra-retinal and perspective cues cause the small range of the induced effect Vision Research Busby A, Ciuffreda K J, 2005 The effect of apparent depth in pictorial images on accommodation Ophthalmic and Physiological Optics Cook N D, Hayashi T, Amemiya T, Suzuki K, Leumann L, 2002 Effects of visual-field inversions on the reverse-perspective illusion Perception Cook N D, Yutsudo A, Fujimoto N, Murata M, 2008 Factors contributing to depth perception: Behavioral studies on the reverse perspective illusion Spatial Vision Erkelens C J, 2012a Contribution of disparity to the perception of 3D shape as revealed by bistability of stereoscopic Necker cubes Seeing and Perceiving Erkelens C J, 2012b Perceived slant of rectangular grids viewed on slanted screens Perception 41 ECVP Supplement, 6 (Abstract) Erkelens C J, Collewijn H, 1985 Motion perception during dichoptic viewing of moving random-dot stereograms Vision Research Gregory R L, 1970 The Intelligent Eye (New York: McGraw-Hill) pp Gregory R L, 1997 Knowledge in perception and illusion Philosophical Transactions of the Royal Society of London B

15 444 J J Dobias, T V Papathomas Hayashi T, Umeda C, Cook N D, 2007 An f MRI study of the reverse perspective illusion Brain Research Held R T, Cooper E A, Banks M S, 2012 Blur and disparity are complementary cues to depth Current Biology Hill H, Bruce V, 1993 Independent effects of lighting, orientation, and stereopsis on the hollow-face illusion Perception Hill H, Bruce V, 1994 A comparison between the hollow-face and hollow-potato illusions Perception Hill H, Johnston A, 2007 The hollow-face illusion: Object-specific knowledge, general assumptions or properties of the stimulus? Perception Hoffman D M, Girshick A R, Akeley K, Banks M S, 2008 Vergence accommodation conflicts hinder visual performance and cause visual fatigue Journal of Vision 8(3):33,1 30 Howard I P, Rogers B J, 2002 Seeing in Depth volume 2 Depth Perception (Toronto: I Porteus) Judge S J, Cumming B G, 1986 Neurons in the monkey midbrain with activity related to vergence eye movement and accommodation Journal of Neurophysiology Julesz B, 1971 Foundations of Cyclopean Perception (Chicago, IL: University of Chicago Press) Matthews H, Hill H, Palmisano S, 2011 Binocular disparity affects perceived depth magnitude despite inversion of depth order Perception Mayhew J E W, Longuet-Higgins H C, 1982 A computational model of binocular depth perception Nature Mon-Williams M, Tresilian J R, 2000 Ordinal depth information from accommodation? Ergonomics Morrison J D, Whiteside T C, 1984 Binocular cues in the perception of distance of a point source of light Perception Papathomas T V, 2002 Experiments on the role of painted cues in Hughes s reverspectives Perception Papathomas T V, 2007 Art pieces that move in our minds an explanation of illusory motion based on depth reversal Spatial Vision Papathomas T V, Bono L M, 2004 Experiments with a hollow mask and a reverspective : top down influences in the inversion effect for 3 D stimuli Perception Pizlo Z, Li Y, Steinman R M, 2008 Binocular disparity only comes into play when everything else fails; a finding with broader implications than one might suppose Spatial Vision Prince S J D, Rogers B J, 1998 Sensitivity to disparity corrugations in peripheral vision Vision Research Regan D, Erkelens C J, Collewijn H, 1986 Necessary conditions for the perception of motion in depth Investigative Ophthalmology & Visual Science Richards W, 2009 Configuration stereopsis: a new look at the depth disparity relation Spatial Vision Rogers B, 2012 Depth constancy and frontal-plane scaling in the absence of vertical disparities Perception 41 ECVP Supplement, (Abstract) Rogers B, Gyani A, 2010 Binocular disparities, motion parallax, and geometric perspective in Patrick Hughes s reverspectives : Theoretical analysis and empirical findings Perception Sherman A, Papathomas T V, Jain A, Keane B P, 2012 The role of stereopsis, motion parallax, perspective, and angle polarity in perceiving 3 D shape Seeing and Perceiving Takeda T, Hashimoto K, Hiruma N, Fukui Y, 1999 Characteristics of accommodation toward apparent depth Vision Research Viguier A, Clément G, Trotter Y, 2001 Distance perception within near visual space Perception Vishwanath D, Blaser E, 2010 Retinal blur and the perception of egocentric distance Journal of Vision 10(10):26, 1 16 Wade N J, Hughes P, 1999 Fooling the eyes: trompe l oeil and reverse perspective Perception Wagner M, Herrnstein W H, Papathomas T V, 2008 Vergence in reverspective: percept-driven versus data-driven eye movement control Neuroscience Letters