Perception, 1994, volume 23, pages 1241-1248 Apparent depth with motion aftereffect and head movement Hiroshi Ono, Hiroyasu Ujike Centre for Vision Research and Department of Psychology, York University, 4700 Keele Street, North York, Ontario M3J 1P3, Canada Paper presented at the Waterfall Illusion Conference, Loch Ness, Inverness, Scotland, 23-25 August 1993 Abstract. Coupling a motion aftereffect (MAE) produced by horizontal shearing motion with a lateral head movement yields apparent depth. In experiments 1 and 2, respectively, the magnitude and the decay time of this apparent depth were measured. In experiment 3, it was found that the stimulus that produced an MAE in experiments 1 and 2 failed to do so when it was viewed while the head moved leftward and rightward and depth was seen. 1 Introduction Unlike the other authors writing in this issue, our primary concern was not with motion aftereffects (MAEs) or with motion perception. Instead, our concern was with depth perception produced by an MAE coupled with a head movement, and with what this depth can tell us about the relationship between motion parallax and motion perception. We started with the idea that both parallactic depth perception and motion perception originate from the same 'front end' motion signal. That is, we thought that these two systems use the same 'front end'. In experiments 1 and 2 we tested the idea that the motion signal that generates an MAE by the motion-perception system can also generate depth by the parallactic depth system, and in experiment 3 we tested whether the hypothesised front end can produce an MAE when 'fatigued' by retinal motion that is coupled to a head movement. 1.1 The basic phenomenon, its explanation, and the questions asked The phenomenon explored in this paper was first noted in 1989 at the ATR Human Information Processing Research Laboratories at Kyoto, Japan by Ono etal (1991). After obtaining an MAE from a stimulus containing shearing motion (see figure la), they found that if the observer's head was moved rightward or leftward (i) adjacent bands in the stimulus appeared in different depth planes (see figure lb), (ii) the apparent motion disappeared or diminished, and (iii) the direction of depth changed with every change in head-movement direction. [This emergence of apparent depth, disappearance of motion, and change in direction of depth can also be seen when the head moves laterally while viewing 'real' motion. See Hayashibe (1993) for this perception under different viewing conditions. The stimulus condition with 'real' motion was used in experiment 3.] The apparent direction of depth reported by Ono et al is consistent with the geometry of motion parallax. In figure lc we can see (for a rightward head movement) that (i)when the motion signal for an MAE (or real motion) and the head movement are in the same direction, the motion signal (or the proximal stimulus of real motion) is equivalent to that produced by a stationary object positioned further away than the fixation point, and the stimulus appears more distant than the fixation point, and (ii) when the motion signal for an MAE (or the proximal stimulus of real motion) and the head movement are in opposite directions, the signal is equivalent to that produced by a stationary object positioned closer than the fixation point, and the stimulus appears less distant than the fixation point.
1242 H Ono, H Ujike When the head moves rightward, as illustrated in figure lc, the apparent direction of depth, therefore, corresponds to the two apparent locations depicted in figures lb and lc. When the head moves leftward (not illustrated in the figure), the motion signal-head movement relationship reverses, and so too does the direction of depth. Thus, every change in head-movement direction is accompanied by a change in the direction of apparent depth. (The magnitude of the apparent depth, indicated by the difference in the two apparent locations depicted in figure lc, is the topic of experiments 1 and 2.) This stimulus differs quite significantly from those used previously to study motion parallax. In past studies, stimulus movement and head movement have been coupled either mechanically (eg Heine 1905) or electronically (eg Rogers and Graham 1979), and thus stimulus movement was bidirectional, and gain (motion signal/head motion) remained constant throughout a given stimulus condition. In the present study, however, the direction of stimulus movement (apparent or real motion) was fixed. One consequence of this was that the direction of apparent depth changed with the direction of head movement. Another possible consequence was a fluctuation in the magnitude of apparent depth when the head accelerated and decelerated, at the beginning and end of each head movement, respectively, because the 'gain' was higher here than when the head was moving through the mid-portion of the head-movement path. (Throughout this paper the term 'gain', with respect to the signal for an MAE Stimulus I Percept (MAE or real motion) (head moves rightward) (a) ; Kb) Figure 1. An illustration of (a) the stimulus used by Ono et al 1991 and in the present study, (b) the resulting percept when the observer moves his/her head rightward, and (c) an explanation of the percept. If the observer moves his/her head leftward (not illustrated), the direction of apparent depth is the opposite of that illustrated.
Apparent depth with MAE and head movement 1243 and a head motion, refers to the ratio between a hypothetical 'motion signal' in the visual system and a head movement.) Despite this change in gain, however, the magnitude of apparent depth remained constant throughout the entire head-velocity cycle. What the observer perceived, therefore, was a given magnitude of depth that changed only in direction (or order) and not in magnitude. Although this lack of expected fluctuation in the magnitude of apparent depth concomitant with the fluctuation in 'gain' within the half cycle of head movement raises a question regarding the integration period of motion parallax, this question is not addressed in this study. We did, however, address the question of 'gain' relative to the magnitude of apparent depth indicated in figure lc with the following operational definition. We incorporated a period of constant head velocity in each half cycle of head movement and varied this velocity across different conditions. We presented the same adaptation stimulus before each of the different head-movement conditions and thus assumed that the magnitude of the underlying motion signal was the same for each of the different conditions. Therefore, when the head moved with different constant velocities in different conditions, the gain was different. In experiment 1 we measured the magnitudes of apparent depth produced by different gains as defined above, whereas in experiment 2 we measured the decay times of the different magnitudes of apparent depths produced by different gains. In experiment 3 we attempted to produce an MAE by viewing a stimulus with apparent depth produced by coupling retinal motion with a lateral head movement. 2 Apparatus, stimulus, and head velocity for the three experiments The apparatus consisted of a Macintosh II computer with 8 bit luminance levels for each of the red, green, and blue phosphors. The stimulus (24 cm x 17 cm: 23degxl7deg) consisted of four bands of 20 cycle sinusoidal gratings, and was presented on a 13 inch Apple Color High Resolution Monitor, positioned 57 cm in front of the observer. The bands (each 24 cm x 4 cm) were separated by a vertical gap of 0.35 cm, and the entire stimulus was surrounded by a 112 cm x 71 cm black board. A fixation point was presented in the centre of the monitor. Head velocity was controlled by a computer-driven bendable guide positioned under the chin. We chose a bendable guide for two reasons. First, observers were unable to rest their chin on the guide and follow it passively, and second, any bending of the guide signaled imprecision in head velocity. (Each observer practiced moving his/her head without bending the guide before the experiments.) When required to view the display with the head stationary, the observer kept the chin on the top of the stationary guide. When required to move the head, the observer actively followed the guide as it moved back and forth, horizontally, through an extent of 30 cm. Throughout the initial and final 5 cm of this movement, the observer's head accelerated or decelerated, respectively. During the middle 20 cm of the movement, the observer's head moved at one of five constant velocities (4, 8, 16, 31.6, or 60 cm s _1 ) in experiments 1 and 2. Although we had originally planned to use these five head velocities in experiment 3 as well, in the event we only used the 16 cm s" 1 head velocity, because we had been unable to produce an MAE in a preliminary study. 3 Experiment 1: Magnitude of apparent depth If the apparent depth resulting from the coupling of an MAE with a head movement is produced by the motion-parallax system, then its apparent magnitude should covary with the gain. That is, with the motion signal that underlies an MAE, the gain is lower in a fast-head-movement condition than in a slow-head-movement condition, and the magnitude of apparent depth, too, should be less. This prediction should hold throughout the range of head velocities for which the magnitude of parallactic
1244 H Ono, H Ujike depth is gain dependent. To test this prediction, we measured the magnitude of apparent depth produced by the motion signal that generates an MAE as a function of head velocity. 3.1 Method Each trial consisted of a 1 min adaptation period and a 10 s test period. During the adaptation period, the bands produced a horizontal shearing motion in which the first and the third bands moved rightward, and the second and the fourth bands moved leftward. The mean luminance and the contrast were 23.8 cd m~ 2 and 85%, respectively, and each band moved with a velocity of 67.2 min s _1. Observers viewed the adaptation stimulus with the head stationary. During the test period, the stimulus remained stationary, and the mean luminance and the contrast were 11.9 cd m~ 2 and 42.5%, respectively. We reduced the contrast of the test stimulus because the apparent velocity of an MAE is reported to be higher with a lower-contrast stimulus (Keck and Pentz 1977). Observers viewed the test stimulus while moving their head at one of the five head velocities described above. After each test period, observers reported the magnitude of apparent depth using calipers. Since the direction of depth varied as a function of head-movement direction, equal numbers of depth reports were made both for the rightward and for the leftward directions. Each of the three observers (one female, 29 years old; two males, 31 and 32 years old) performed 12 trials per head-velocity condition, for a total of 60 trials. The trials were completed in six blocks; within each block 2 trials from each of the five head-velocity conditions were represented. The presentation order of head-velocity conditions was random within each block. i ^ MU x l 10 100 Head velocity/cms -1 Figure 2. The magnitude of apparent depth produced by the signal that generates an MAE as a function of head velocity for the three observers in experiment 1.
Apparent depth with MAE and head movement 1245 3.2 Results and discussion The results were consistent with the expectation that a slow head movement would yield a larger apparent depth. The mean magnitude of apparent depth decreased monotonically as the head velocity increased, as is shown in figure 2. The differences between conditions were highly statistically significant across subjects {F 4tl0 = 8.46, p < 0.01) as well as within subjects: for subject HU F 4j55 = 37.20, p < 0.01; for subject MU F 455 = 39.55, p < 0.01; and for subject QY F 4>55 = 13.85, p < 0.01. The results indicate that the apparent magnitude of depth covaried with the gain in accordance with our predictions. They strongly suggest that coupling the motion signal that underlies an MAE with a head movement generates depth perception by the parallactic system in the same manner as does coupling the motion signal from the retina with a head movement. 4 Experiment 2: Decay time of apparent depth The motion signal that generates an MAE is known to decay over time and so too, therefore, should the apparent depth produced by it when it is coupled with a head movement. Moreover, the time for this apparent depth to disappear should be shorter with a fast head movement than with a slow head movement. This prediction is based on the general belief that the threshold of parallactic depth is thought to be determined by gain. That is, the threshold is usually specified in terms of equivalent disparity which, as defined by Rogers and Graham (1979), is the difference in relative angular displacement per head movement equal to the interocular distance, which is analogous to what we call gain in this paper. As the motion signal decays (ie signals lower velocity or a smaller displacement), the gain decreases, and with a faster head movement this decreasing gain would reach the threshold sooner than with a slower head movement. To test this prediction we measured the decay time of the apparent depth produced by the same motion signal used in experiment 1 with different head velocities. 4.1 Method The adaptation stimulus, the test stimulus, the observers, and the adaptation period were identical to those of experiment 1. During the test period, however, observers viewed the test stimulus while moving their heads at one of the five head velocities described in section 2, until they no longer saw depth. Each of the three observers performed 8 trials per head-velocity condition, for a total of 40 trials. The trials were performed in eight blocks and within each block 1 trial from each of the five head-velocity conditions was represented. To reduce the effect of motion adaptation across the trials, a 4 min rest period separated the trials. The presentation order of head-velocity conditions was random within each block. 4.2 Results and discussion Again, the results were consistent with our expectation: the decay time of apparent depth generated by a fast head movement was shorter than that generated by a slow head movement. The mean decay time for the apparent depths decreased monotonically as the head velocity increased, as is shown in figure 3. As in experiment 1, the differences between conditions were highly statistically significant across subjects (F 410 = 28.11, p < 0.01) as well as within subjects: for subject HU F 4t3S = 56.90, p < 0.01; for subject MU F 4 35 = 177.06, p < 0.01; and for subject QY F 4 35 = 60.09, p < 0.01. Also, consistent with the hypothesis, all three observers reported that with a fast head movement, once the perception of depth had disappeared, the perception of motion lingered. This lingering MAE was seen when the apparent depth had disappeared and the head was held stationary. This report is consistent with the idea that the parallactic depth threshold is gain dependent,
1246 H Ono, H Ujike because for depth to be seen with a fast head movement a larger retinal motion signal is required than with a slow head movement. To state this differently, a small motion signal is not used for depth with a fast head movement, because the threshold is determined by the gain. 80 HU 60 40 20 OH 80 r 60 r 40 r 20 80 MU 60 40 20 r 0 1 10 Head velocity/cm s" 100 Figure 3. Decay time of the percept of apparent depth as a function of head velocity for the three observers in experiment 2. 5 Experiment 3: Motion perception and motion parallax The results from experiments 1 and 2 suggest that motion perception and parallactic depth perception share a common front end. Moreover, one can postulate, analogously to a 'motion detector' (eg Barlow and Levick 1965), that in addition to relaying motion signals this front end, when fatigued, is the source of the motion signal for MAEs. We therefore attempted to obtain an MAE from a stimulus which provided a retinal motion signal, but which did not produce a perception of motion. This was done by 'adapting' the subject to a stimulus containing shearing motion, while moving the subject's head leftward and rightward. Although with the head movement depth, rather than motion, is seen (see section 1.1), the retinal motion is almost identical to that in experiments 1 and 2, owing to the use of a fixation point. The same 'fatiguing' of the front end was expected, and consequently so was an MAE. 5.1 Method We added two new observers (one female, aged 31 years, and one male, aged 46 years). Each trial consisted of a 5 min adaptation period and a test period. The adaptation stimulus was identical to that in experiments 1 and 2 with the exception that the bands moved with a velocity of either 2 min s" 1 (one observer) or 4 min s _1 (four observers). We reduced the velocity of the adaptation stimulus to make certain
Apparent depth with MAE and head movement 1247 that one observer saw depth among the bands but did not see motion during the adaptation period. Observers viewed the stimulus while moving their heads leftward and rightward at a velocity of 16 cm s" 1 and while maintaining fixation on the fixation point in the centre of the screen. The test stimulus was identical to that of experiment 1 (ie no shearing motion), and the observers viewed it with the head stationary. The observers' task was to report whether or not they saw an MAE. Each of the five observers performed at least 3 trials. 5.2 Results and discussion Although the retinal motion in this experiment was very similar to that in the other two experiments, and the adaptation period was much longer than in either of the other two experiments, no MAE was obtained. The results do not necessarily argue against the hypothesised common front end for the two perceptions, but the front end cannot be a simple bottom-up transmitter of motion signals as we had envisioned a motion detector to be. To maintain the idea of the common front end requires a 'feed forward' mechanism from the depth-perception level to the front end. The reasonableness of such a postulate is difficult to judge based on the finding of this study. Walter Ehrenstein suggested, at the Waterfall Illusion Conference in 1993, that a possible explanation for this result may lie in the distinction between reafference and exafference (von Hoist and Middelstaedt 1950). In this light, our findings suggest that (i) 'exafferent motion signals' (ie motion signals not coupled with head movement) yield perceptions of motion and MAEs, and that (ii) 'reafferent motion signals', either from an MAE or from real motion (ie motion signals coupled with head movement) yield neither perceptions of motion nor of MAEs. How to model such a system, however, is not yet clear to us. 6 General discussion The results of experiments 1 and 2 closely link the content of a 'reafferent motion signal' to whatever is contained in a 'reafferent MAE'; however, the results from experiment 3 raise questions regarding the origin of the signal. But what does the signal(s) for an MAE, reafferent or exafferent, contain? Is the perception of an MAE dependent on the presence of a velocity signal, a displacement signal, or both? In physics, the presence of a velocity signal infers the presence of a displacement signal because velocity is defined as displacement/time. In a biological system, however, it can be argued that velocity and displacement (two locations) are coded separately (MacKay 1976; Regan and Beverley 1984) as in a car having a speedometer and an odometer. Therefore, based on the phenomenological experience of an MAE (ie an object which appears to move, yet not go anywhere) it is reasonable to conclude that an MAE is dependent on velocity signals and not on displacement signals. This premise, coupled with our finding that parallactic depth can be produced from an MAE, suggests that parallactic depth perception is also dependent on velocity signals rather than on displacement signals. However, we are reluctant to draw this conclusion as yet, because the inference from the phenomenological experience of an MAE is somewhat questionable. This is so because an MAE can be nulled with real motion, which presumably contains both velocity and displacement signals, yet there is no 'residual' perception of displacement. Perhaps, a better understanding of motion parallax must await a better understanding of the nature of MAEs, and an explanation of the failure to obtain an MAE in experiment 3 may be a first step to that end.
1248 H Ono, H Ujike Acknowledgements. Very capable editorial help by Alistair Mapp and preparation of the figures by Audrey Smith-Mapp are acknowledged. This study was supported by Grant A0296 from the Natural Sciences and Engineering Research Council of Canada, a grant from the Institute for Space and Terrestrial Science, and NATO grant RG0067/89. The rewriting of this paper to incorporate the reviewers' comments was done while the first author was a visiting researcher at the ATR Human Information Processing Research Laboratories, Kyoto, Japan. References Barlow H B, Levick W R, 1965 "The mechanism of directionally selective units in rabbit's retina" Journal of Physiology 178 477-504 Hayashibe K, 1993 "Head movement changes apparent depth order in motion-parallax display" Perception 22 643-652 Heine L, 1905 "Uber Wahrnehmung und Vorstellung von Entfernungsunterschieden" Albrecht von GraefesArchivfurKlinische undexperimentelle Ophthalmologic 61 484-498 Hoist E von, Middelstaedt H, 1950 "Das Reafferenzprinzip" Naturwissenschaften 37 464-476 Keck M J, Pentz B, 1977 "Recovery from adaptation to moving gratings" Perception 6 719-725 MacKay D M, 1976 "Perceptual conflict between visual motion and change of location" Vision Research 16 557-558 Ono H, Shioiri S, Sato T, 1991 "Parallactic depth perception after motion adaptation" Investigative Ophthalmology and Visual Science 32 1052 Regan D, Beverley K I, 1984 "Figure-ground segregation by motion contrast and by luminance contrast" Journal of the Optical Society ofamerica A 1 433-442 Rogers B J, Graham ME, 1979 "Motion parallax as an independent cue for depth perception" Perception 8 125-134 p 1994 a Pion publication printed in Great Britain