Judgments of path, not heading, guide locomotion

Transcription

1 Judgments of path, not heading, guide locomotion Richard M. Wilkie & John P. Wann School of Psychology University of Reading Please direct correspondence to: Prof J. Wann School of Psychology, University of Reading, Earley Gate, Whiteknights, Reading, RG6 6AL, UK Webpage: Phone: +44(0) Fax: +44 (0)

2 Abstract To steer a course through the world we are almost entirely dependent upon visual information, of which a key component is optic flow. In many models of locomotion heading is described as the fundamental control variable, however, it has also been shown that fixating points along or near your future path could be the basis of an efficient control solution. Here we aim to establish how well observers can pinpoint instantaneous heading and path, by measuring their accuracy when looking at these features while traveling along straight and curved paths. The results showed that observers could identify both heading and path accurately (~3º) when traveling along straight paths, but on curved paths they were more accurate at identifying a point on their future path (~5º) than indicating their instantaneous heading (~13º). Furthermore, whilst participants could track changes in the tightness of their path, they were unable to accurately track the rate of change of heading. In light of these results we suggest it is unlikely that heading is primarily used by the visual system to support active steering. Keywords human locomotion, path, heading, optic flow, eye movements, active gaze 2

3 Introduction Control of locomotion is fundamental to exploring and interacting with the world around us, whether we are walking, running or driving a car. Such control is usually quite effortless, even when we are traveling at artificially high speeds. Gibson initially conceived that navigation through the world was achieved by ensuring the focus of expansion (FoE) of optic flow lay in the direction you wished to travel. This strategy was simple, but unfortunately it did not take into account the influence of eyemovements, which introduce an additional rotation component into the flow field at the retina. As a result considerable effort has been directed towards solving how observers might extract the FoE from a retinal flow-field after it has been transformed by an eye-rotation (e.g. Longuet-Higgins & Prazdny, 1980). The agenda was set for a series of heading experiments that defined the conditions under which accurate judgments of locomotor heading could be made (Warren, Morris & Kalish, 1988; Warren & Hannon, 1990; Royden, Banks & Crowell, 1992; Royden, Crowell & Banks, 1994; Banks, Ehrlich, Backus & Crowell, 1996; Warren, 1998). These studies tell us that observers can recover heading on linear paths when eye-movement signals are available to the visual system (or eye rotations rates are low). A number of experiments extended these investigations to examine direction of self-motion judgments when traveling on circular paths, the more general case in locomotion. There are two potential classes of information available to the observer in this case: there are the points on the ground that the observer will pass over if they maintain their current curved trajectory, which we will call the locomotor path; and also, tangential to this path (and usually straight ahead of the observer), is the instantaneous heading. Stone & Perrone (1997) observed accurate (<~4º) instantaneous heading judgments in 5 out of the 6 highly trained participants, while Cutting, Wang, Fluckiger & Baumberger (1999) found that instantaneous heading seemed to be predominantly used by participants to judge their future course in the world, although in general performance was poor. Warren, Mestre, Blackwell & Morris (1991) found that participants could make quite accurate forced choice judgments of their future path (<~2º), although confusingly they used the term 3

4 circular heading 1. A number of the paths used by Warren et al., however, were weakly curved with radii of 120m 320m which would only have rotated the observer through 3º-7º (in world co-ordinates) during the trajectory, thereby presenting a flow field that was slightly off-radial. On European roads, drivers frequently encounter bends of less than 80m radius (the tightest turn used by Warren et al) and for turns into side-roads these may be below 20m radius. Cyclists routinely take the latter at speed. The task of Warren et al. required forced choice judgments with respect to a probe presented 16m ahead, at four constant stimuli settings (0.5, 1, 2, 4º) around the path. In principle this psychophysical method requires a very large number of trials to reliably estimate the perceptual threshold. We would also argue that a post-hoc forced choice method does not confirm that observers can perceive their path to within 2º of accuracy during locomotion, merely that they can apply a suitable heuristic after the event to lift performance above chance to at least 75% correct. This is a general issue when psychophysical judgments are recorded after the movement phase has ended, as they cannot confirm the accuracy of an instantaneous percept of heading or path during actual locomotion. Wann & Land (2000) have alluded to this by questioning whether heading is a post-hoc percept in the sense that it may be one that can be recovered by observers if they are required to do so, but that is not actually used in active naturalistic tasks (p. 324). We wished to revisit the issue of path perception with a more direct, real-time estimate of what a driver might be able to perceive. In previous studies that have examined circular paths there have been widespread differences in fixation requirements (gaze fixated and static, fixated and tracking, or free gaze) and the visual environments used (cloud of dots, fronto-parallel or ground planes). For free gaze with a stable ground plane (which is the favored man-made locomotor environment) active gaze can be used to sample appropriately from the optic array in order to guide steering along a curved path. If an observer fixates a point they wish to pass through, that is not on their current path, then the retinal trajectory of ground elements will be curved, and the direction of curvature can indicate understeer or oversteer, relative to the desired path. If they fixate a point on 1 Although in common language we may say I was heading round the bend, past research has defined heading as the direction that the observer is travelling at a particular instant, which would be the tangent to a curved path. The path is comprised of points that the observer will pass through, if they maintain their current steering response, but which they are not as yet heading directly towards. 4

5 their future path the retinal flow lines remain straight, though not necessarily radial (Kim & Turvey, 1999; Wann & Land, 2000; Wann & Swapp, 2000). Instead of requiring the decomposition of the retinal flow field to return a heading estimate, this approach requires active gaze to select a point on the required path and then use retinal flow to classify the current direction of locomotion as an understeer or oversteer, relative to the desired path. Van den Berg, Beintema & Frens (2001) have questioned this account and demonstrated that the addition of asymmetrical speed vectors (bi-radial flow) can lead observers to report that they are on a curved path. There are also a number of other conditions that can lead observers to (mis-)perceive their path as curved, such as eye-rotation (Royden et al, 1994) or vestibular stimulation. However, if an observer knows that they are on a curved locomotor trajectory (e.g. they have just turned their steering wheel or handlebars), the conditions outlined above would allow them to identify a future points on their path. This highlights an important difference between path perception and the use of heading. If the retinal image requires significant processing, to recover heading and support steering, then eye movements can only ever complicate the image pattern (Lappe, Bremmer & van den Berg, 1999). In contrast we propose that active gaze can be used to identify the future path (Wann & Land, 2000), in which case eye movements are part of the mechanism for extracting locomotion information from the visual scene. Here we examine judgments of heading and path under conditions similar to those experienced when traveling in a vehicle. If you have been looking down at your map, then glance up at the scene, is it possible to rapidly determine your path or instantaneous heading? To explore this we tracked the eye movements of our participants and then prompted them to look to their heading or path, during active locomotion, rather than via post-trial evaluation. The first experiment presents the simplest case of heading detection, of traveling on a straight path, fixating an eccentric target at the horizon, and making a saccade to identify either the FoE of the flow field or a point on the future path on the ground. We repeated these trials when fixating an eccentric target over the ground, and then compared performance with the classic heading-judgment task of traveling on a linear path and rotating gaze to track and maintain fixation on a target moving in depth. Once again the response requirement was to suddenly glance across and fixate locomotor heading (which if 5

6 accurate would reveal a veridical FoE: Figure 1) or onto the future path. The second experiment looked at equivalent judgments when traveling on a curved path, which would ordinarily require a driver to modify their direction of heading (Figure 2a). Traveling on a linear path while fixating an eccentric target and traveling on a curved path, both introduce a rotation into the flow field. In the former, however, the flow rotation is directly correlated with the degree of gaze motion, whereas in the latter it is not. We hypothesize that the extra-retinal signals from gaze motion will allow observers to recover instantaneous heading for the straight line path (Royden et al., 1992), but that the percept of heading will be lost for a curved path. [Insert Figure 1 & 2 around here] It could be argued that it is the rate of change of heading that provides the most useful information for controlling steering. For example, heading that is changing at a constant rate ensures that a path of constant curvature is followed, and such information could provide an approximate estimate of future path curvature and position. To explore this argument, the third experiment presented either a path of changing curvature, with a stable heading location (Figure 2b), or a changing heading with a path of fixed curvature (Figure 2c). In these cases we allowed participants to use free gaze and tested whether they could match the rate of motion of a visual probe with a changing path or a changing heading. General Method Each trial presented the viewer with locomotion (at 8ms -1 ) through a visual environment containing a ground plane textured with gravel (Figure 3). The image generation platform was a PC with Dual Xeon Processors (Intel Corporation, Santa Clara, CA) running tailor-made software with Direct X libraries under Windows 2000 (Microsoft Corporation, Redmond, WA). Images were generated at a frame rate of 50 Hz, and were projected onto a large screen (2 x 1.5m) that was a meter away from the observer and therefore subtended a total angle of 90 x The projection system was Hitachi Liquid Crystal projector (1024x768; 0.1º per pixel). The observers used both eyes to view the non-stereo image (Bi-ocular viewing). Gaze direction of the left eye was monitored at 60Hz using an ASL 504 system. A repeat-fixation calibration 6

7 routine with our display setting established an effective resolution for this of ±0.32º. A time-based calibration also established the transmission lag in providing eyeposition data was 120ms. Practice in Experiment 3 was given with the heading location marked, and the future path drawn to ensure that the participants correctly understood the more complex task of matching the rate of change of heading or path, whilst also generating an appropriate motor mapping for the control device. [Insert Figure 3 around here] Experiment 1: Linear Paths If you are traveling on a straight path then the optic flow field is comprised of ground elements that appear to radiate outwards from a distant focus of expansion (FoE) which is coincident with your heading (Gibson, 1958). If your gaze is stable then the FoE will also be present in the flow field on the retina. The first condition in this experiment presents the simplest of heading detection tasks. Participants were traveling along a straight path, with gaze stable, but fixating a cross offset to the right or left of their heading at the horizon. We then measured the accuracy with which they could identify the FoE and saccade sideways to look at it. We then repeated these trials, but after fixating the cross we asked participants to look towards a ground point on their future linear path. (Figure 1a). The second condition was identical, except that the participant was required to fixate a static cross lying above a point on the ground 16m ahead before making the saccade to look at the FoE or path (Figure 1b). The third condition was similar, except it required visual tracking of a fixation cross that was attached to the ground plane (Figure 1c) before making a saccade towards either the heading direction or a point on the path. To keep a ground feature fixated, gaze has to rotate downwards and sideways and this introduces a rotation into the flow presented to the back of the eye which masks the true FoE (Warren & Hannon, 1990). The third condition tests whether participants can glance up from their tracking either to their heading or a point on their linear path. If they executed an accurate saccade, then once their gaze landed and was stable, they would perceive the FoE on the horizon. Looking to your heading on the horizon or a point on your path, in 7

8 essence are both heading measures. Minor differences may occur when looking at a ground point on the linear path, because ground elements would be moving past their gaze as it landed and optokinetic nystagmus (OKN) may bias gaze stabilization. The second condition (where gaze was initially stable and fixating 16m ahead: Figure 1b) was included so that participants had to initiate their saccade from a similar eye position as for the ground tracking condition. Method Six participants (CB, CH, DP, DF, JB, RW) with normal or corrected-to-normal vision took part in the experiment. Locomotion was simulated (at 8ms -1 ) across a ground plane on one of three predefined headings to the left or right of the median sagittal plane of the body or body-line (±6, 10, 14º). Three fixation conditions were presented: stable gaze fixating horizon (Horizon), stable gaze fixating 16m ahead (16m), and tracking a fixation cross on the ground eccentric to the direction of motion, and moving from 28.5m to 16m away (Ground). To prevent the use of a learnt association between the fixation point and heading the angle of the scene camera was also offset by ±0, 5 or 10º. After 1.5s the participant heard a tone that told them look away from the fixation point towards their heading on the horizon, or a point on the path on the ground approximately 16m ahead. Trials lasted 2.5 seconds and there were 18 trials for each of the fixation and saccade locations (108 in total). All participants were given 6 practice trials within each condition to familiarize themselves with the fixation requirements and the procedure of changing point of gaze, but no trial-by-trial feedback was provided for these tasks. Results and Discussion Participants could identify the FoE and saccade to it from a fixed gaze position with an accuracy of ~3º. The were no differences in performance between judgment type (F(1,5)=.57, ns) or fixation condition (F(2,10)=1.86, ns) demonstrating that observers could rapidly locate their heading even when their gaze was rotating to fixate a ground feature (Figure 4). This level of precision is equivalent to previous studies that have used active post-hoc pointing which confirms the validity of the gaze tracking method employed here. A linear course means that path and heading are coincident, with heading lying at the end of the points on your path. Locomotion along a straight 8

9 path, however, is not the general case, and to steer successfully at speed we must be able to control changes in our direction of motion over time. Circular paths cause a divergence between path and heading, with the points on the future path curving away from your current direction of motion. In the next experiment we address whether path and heading can be accurately perceived for curved trajectories. [Insert Figure 4 around here] Experiment 2: Curved Paths We routinely need to maintain straight paths in critical settings such as on highways, but we also often need to change our direction of travel and steer a curved path, to leave the freeway, or to steer within an open space. The general task in locomotion is maintaining a (curved) trajectory and predicting future path, of which a straight trajectory is the limit case of zero curvature. In this experiment we address the issue of whether the perception of instantaneous heading generalizes to the broader class of curved trajectories. In common with the ground tracking condition in experiment 1 (Figure 1c) locomotion along a curved path introduces rotation into the retinal flow field (Figure 2a). An important difference, however, is that the flow field rotation in experiment 1 was directly correlated with the gaze motion signals, whereas for a curved path it is not. This means that for a curved path an estimate of the instantaneous heading must be recovered from the statistical properties of the flow field rather than through the addition of extra-retinal information regarding gaze motion. In contrast, an estimate of the curved path could be gleaned from the retinal flow pattern or the gaze motion if an appropriate fixation is made (Kim & Turvey, 1999; Wann & Land, 2000; Wann & Swapp, 2000). In experiment 2 we ask participants to glance to their heading or glance to their path, equivalent to the task presented in experiment 1, but with curved paths to test the extent to which heading information is available. In this setting the instructions to participants become important. In pilot trails, when participants were asked to look to where they were heading, they were inclined to look towards their future path, which would have resulted in large errors. To help our participants understand the task, instantaneous heading was described to them as the point towards which they would travel if they suddenly hit ice on the road and lost traction, and also as the tangent to their current 9

10 curved path. The implications of this instruction are considered in the general discussion. Method Six participants (AP, CW, EF, JF, KW, SH) with normal or corrected-to-normal vision took part in the experiment. Curved paths were presented with a locomotor speed of 8ms -1. Paths were simulated which could curve either left or right of the observer s body-line (the median sagittal plane of the body) and had a curvature of either 0.015m -1 or 0.018m -1 with an additional random factor of m -1. These trajectories were tighter than the lower radius used by Warren et al (55-66m vs. 80m) and are equivalent to those encountered in everyday locomotion. Trials lasted for 8.8s and would have resulted in the observer moving through a total bend-angle that ranged from 62º and 72º with a rate of change of direction between 7.1º/s and 8.2º/s, respectively. During each trial the observer was asked to look to a point on their future path or to their instantaneous heading, without any visible indicator being present. When making judgments we asked participants to look to their heading on the horizon, to minimize the effects of OKN, which were directly assessed in a control condition described in the results section. With a fixed projection surface, in principle, the tangent to the curve would always be the center of the screen (in the natural world the same is true for the center of a car windshield). To eliminate this potential confound the world camera, and hence the tangent to the curve, was offset from the screen center by ±10º or ±15º. The offset remained constant for any particular trial, changing only at each new trial trajectory. In Path Main the observer was instructed to look at a point, of their choosing, that lay on their future path. When they heard a tone they were to look up to fixate their instantaneous heading on the horizon and then look back down again to their path. To counterbalance, during Heading Main, observers were instructed to look at their heading until they heard the tone, at which point they should look down to fixate their path and then look back at their heading. The initiation tone was played twice during each trial and eye-movements were recorded throughout trials, which could be compared with the actual heading or path at key frames. From these comparisons a set of angular gaze errors (in degrees) were calculated for path and heading fixations. The calculation procedure is outlined in Figure 3. 10

11 [Insert Table 1 & Figure 5 around here] Results and Discussion The participant used their gaze to make both path and heading judgments. Gaze error was calculated in two ways: root mean squared (RMS) gaze error was calculated as an overall measure of precision, and constant error (CE) as a measure of bias. There were no significant differences between judgments made before, after or at the prompting tone (Table 1) for heading (F(2,10) =.26, ns) or path (F(2,10) =.28, ns) so data was collapsed across these. The RMS errors demonstrate that observers were more accurate at looking at their path than at instantaneous heading (Figure 5: t(5)=8.06, p<.01). It has been calculated that humans require a translational heading accuracy in the order of 1-3º of visual angle to avoid obstacles when moving at speed (Cutting, 1986). For the specific situation of driving on a UK road we can also estimate acceptable error. The average UK car has a width of 1.5m with the lanes in main roads approximately 3m wide, dropping to 2.4m for side roads. At the velocity used in these experiments (8m/s = 18mph) an error of 5.4º would result in the car encroaching on the road center within 1s. Error in judging heading position (12.6º) was significantly greater than this value (t(5)= 4.13, p<.01), and can be taken as evidence that heading information is not perceived with sufficient accuracy for effective control of locomotion. When the predominant point of gaze was towards the path the mean error (5.24º) was equivalent to our threshold of 5.4º, and a one-sample t-test finds the path error not significantly different from Cutting s value of 3º (t(5)=2.4, ns). It was possible that the observers may have made path judgments by looking at a very near point where flow was not greatly curved, and the task would have been easier. To discount this hypothesis we examined the characteristics of path judgments. Gaze was directed down from the horizon by ~8.6º and laterally from the midline by ~11.6º, suggesting that path judgments where made to points on the path that were both curved and distant. These judgments equate to fixating a point 13.1m ahead, giving an average look-ahead time of 1.63s, commensurate to look-ahead distances when steering (~1-2s : Land & Lee, 1994; Wilkie & Wann, 2003) and similar to our fixation distance in experiment 1. 11

12 When gaze lands upon an area of ground that the participant judges to be their path, it is possible that gaze is dragged down by the moving ground elements. We assessed whether optokinetic nystagmus (OKN) could explain differences in gaze behavior for the two experimental conditions. We presented the same stimuli as in experiment 2, but with a fixation cross present at the actual heading or path. Participants were asked to maintain fixation upon this location during trials even when, after 2 seconds, the cross was removed. We found RMS deviations of 0.35º when looking at heading on the horizon, and deviations of 1.13º when looking at their path on the ground. Fluctuations in gaze may therefore have increased the error to path judgments by around 1º, but do not explain the erroneous heading judgments. The apparent errors in heading judgments could be explained by observers making precise but mis-localised judgments, however, the variability (standard deviations) of the heading judgments were also significantly greater than for path judgments (t(5)=5.74, p<.01), arguing against simple mis-localisation. Experiment 3: Rate of change of heading and changes in path curvature In experiment 2 we demonstrated that, during a curved trajectory, participants can make some estimate of their instantaneous heading, but their accuracy is not sufficient for this to be used to control locomotion. By comparison participants judgments of their future path was significantly more accurate, even though this estimate was more prone to contamination by OKN. It could be the case in a natural scene, however, that path curvature is estimated by judging the rate of change of heading. To assess this we created a situation where the heading direction moved across the screen, while the path curvature remained constant and compared this with a situation where the heading direction remain constant, but the path curvature increased or decreased (Figure 2b&c). Both the rate of change of heading and the change in path curvature introduce an acceleration component into the flow field and we scale these to be of a similar order. Although we used a gaze response in experiments 1 & 2, we found that in this experiment, across these type of moving flow fields, it was difficult to measure smooth gaze tracking. We therefore allowed participants free gaze and tested whether 12

13 they could move a visual probe via a steering wheel to match changing heading or a point on the changing path. Method Six participants (HH, CB, DP, DF, CH, RW) with normal or corrected-to-normal vision took part in the experiment. Two types of stimuli were generated to test perception of a changing heading or path. To test an observer s ability to perceive rate of change of path we simulated locomotion along paths of tightening curvature (the beginning section of a spiral) over a textured ground plane. In both cases locomotor speed was 8ms -1 and trials lasted 7.5s. The curved paths were initiated with a standard 5.4º/s rate, with one of three accelerations selected randomly in different trials (0.013, 0.02, 0.026º/s/s) resulting in total angular displacement on the screen of 7.5º, 11.25º & 15º respectively (Figure 2b). It should be noted that in this situation heading does not change, remaining straight ahead of the observer, in line with their mid-line. The tightening of the path presents on the projected image as the centre of the path yawing sideways at a smooth velocity. A visual probe was present, 16m in front of the observer s position, and their task was to keep this above the future path, and match the change in position as the path tightened. The probe started in the correct position over the path, and the rate of rotation of the probe was controlled with a force-feedback steering wheel. The steering wheel acted as a rate-control device, where an increased angle of turn increased the rate of probe motion. Without participant input path judgments would have gradually become more erroneous over the course of the trial. We calculated the pursuit gain for each trial as a ratio of the velocity of the probe to the projected velocity of the center of the path (see Fig. 6). This measure would not be affected by errors in the precise localization of the center of the path (which was assessed in the previous experiment), but reflects sensitivity to the rate of change of the path. To test an observer s ability to perceive rate of change of heading we started with simulated locomotion along a path of constant curvature. In this situation neither heading nor path are changing, however we then rotated the camera view independent of the locomotor path (Figure 2b&c). It was then the task of the observer to keep the position of the probe in line with the heading, and match the rate of change of position of the probe with the change in heading. The probe was located vertically at the 13

14 horizon, and started horizontally in the correct position above the heading. Heading was changed at an accelerating rate (0.005º/s/s, 0.01º/s/s), matched to be the same angular displacement on the screen as that used in the rate of path manipulation, this gave a similar velocity function for tracking with the probe and the pursuit gain was calculated in an identical manner to the path tracking. Figure 6 presents prototypical functions for the motion of the path or heading and it can be seen that the matching of total displacement results in a very similar pursuit task in both conditions. Participants were given 6 practice trials with displays showing the path and heading marked out. This ensured that the participants understood the instructions, and in particular completed the heading task to the best of their ability. [Insert Figure 6 around here] Results and Discussion We simulated circular paths with changing heading, or changing curvature paths with constant heading to examine whether the rate of change of heading or path could be perceived and tracked. To match the perceived motion observers had to rotate the steering wheel, rather than just hold it at a fixed angle. We have already established in experiment 2 that the heading judgments may be inaccurate by as much as 12 o, but in this case we are not concerned with the judgment of heading location, but rather its rate of change. Figure 7 presents the pursuit gain of the participants tracking of the path/heading changes. The gain measure is independent of location errors. It is clear that observers managed to match the path motion quite accurately, but were significantly poorer at matching the changing heading (t(5)=21.9, p<.001). [Insert Figure 7 around here] General Discussion In this paper we tested the ability of the human visual system to perceive heading and path during simulated locomotion. We tracked observer performance, using either their point of gaze or control of a visual probe, to test their ability to judge heading or path during locomotion, rather than via post-trial evaluation. 14

15 In locomotor control, maintaining a straight-line path should not be a difficult perceptual task. It may be achieved by judging heading from flow (Warren & Hannon, 1990) or on the basis of other visual direction cues (Rushton, Harris, Lloyd & Wann, 1998) or through a combination of flow with other information (Warren, Kay, Zosh, Duchon & Sahuc, 2001). It can be more difficult if gaze is averted to track a moving target, such as looking to a roadside sign, particularly if the sign requires close attention (Wann, Swapp & Rushton, 2000). But the presence of a correlated gaze motion signal seems to aid the judgment of heading in such situations. In the first experiment we included the classic heading-judgment task of traveling on a linear path, and fixating an eccentric target. An everyday response to the road-sign dilemma is to alternate gaze and briefly saccade forward to confirm that the radial flow field is centered around the required direction of travel. In this study we used an equivalent mode of response and the pattern of results demonstrated that observers could look instantaneously to their heading with an accuracy of ~3º even if their gaze was rotating to track a feature attached to the ground. So for locomotion in a straight line observers could accurately locate their heading at a glance, which agrees with previous findings, and also confirmed the validity of the gaze tracking method. In the second experiment we examined the more general case of curved paths, where rotation occurs in the flow field, but this is not primarily due to gaze rotation or accompanied by a correlated gaze rotation signal. Fajen & Warren (2003) studied participants walking curved paths towards a target and proposed that participants acted to null the angle between the target and their instantaneous heading (β). They then extended this study (Fajen & Warren, 2004) to examine the more difficult task of intercepting moving targets and proposed that for fast moving targets this may switch to a strategy of nulling the rate of change of β. In both cases participants would need an accurate estimate of their instantaneous heading, though in the latter case an estimate of the rate of change of heading could suffice. Experiment 2 tested heading judgments when traveling on a curved path whereas Experiment 3 tested judgments of the rate of change of heading. In Experiment 2 we found that errors in judged heading position (12.6º) far exceeded those required for safe control of locomotion and this can be taken as evidence that heading information cannot be perceived with sufficient accuracy for effective control of locomotion. In contrast the saccades from the 15

16 horizon to a ground point on the future path were significantly more accurate. This illustrates that locomotor flow lines (Lee & Lishman, 1977) can be perceived accurately when gaze is averted and that observers can pinpoint with a saccade the central path within the flow lines to approximately 5º. One suggestion during review was that the instructions to participants could have influenced their strategy in attempting to identify heading. The verbal explanation that instantaneous heading is the the tangent to the curved path may lead them to try and estimate the tangent. In order to do this participants would need to use the ground flow information to recover the path trajectory, extrapolate the path backwards to under their feet and then project this forwards to the horizon. There is no previous research to suggest that this is a common strategy, but if it was adopted, that in itself would highlight our point that a curved path is well defined by ground flow, but by comparison instantaneous heading is not. If a percept of instantaneous heading popped-out from the ground flow-field there would be no need to use a secondary strategy such as path extrapolation. Anyone who routinely cycles or drives a vehicle can reflect upon our findings from experiment 2: In steering a bend, you will presumably be confident about where you will be on the roadway some 1-2secs later (path), but you may also introspect upon whether you can perceive your instantaneous heading (or it s rate of change) as you steer. We believe that heading is not apparent and possibly one of the few ways of recovering it is to use a feature such as the front of your car or your handlebars as a visual cue to instantaneous direction of travel. In support of this suggestion experiment 3 directly assessed the perception of rate of change of heading (with a path of fixed curvature) and found it was not perceived accurately. In contrast participants accurately matched the rate of change of a tightening path. Stone & Perrone (1997) used curved paths and rotation rates similar to ours and found quite accurate performance on heading judgments (~4º) compared to our participants (~13º). A critical difference may be that the participants of Stone & Perrone received training before they were able to make reliable heading judgments and they note that despite these explanations, the task is difficult and casual observers are generally not able to perform well (p. 577). This is very much in line with our anecdotal report in the methods for experiment 2: Our participants, who routinely steered or cycled around the world, found it easy to comprehend what we meant by their future path, but required a more elaborate explanation to understand the concept of instantaneous 16

17 heading. We do not deny that with experience and training participants may be able judge heading in many settings, in line with the post-hoc percept suggestion of Wann & Land (2000), but we doubt that they do so in natural settings. We do not believe that heading is a percept that readily pops-out from the flow field of a complex trajectory and we suggest that this lack of salience also casts doubt upon its role in locomotor control. Considering other differences with previous research, it may be noted that the errors we observe in path judgments (~5º) are greater than might be expected from previous research on heading judgments (e.g. Warren et al., 1991). As outlined in the results section of experiment 2 our gaze point measures may be inflated to some degree by observer OKN, particularly as our observers were trying to fixate a location on a moving homogenous textured surface. In the real world it is rare that locomotion occurs in a situation where there are no visible features on the ground that can be easily identified and fixated (for instance road markings, rocks or holes, or other shadowed surfaces). But we would also argue that rapid, real-time response measures cannot be directly compared with post-hoc forced-choice judgments. Previous psychophysical studies have often used a 75% correct threshold to estimate the accuracy of judgments. It is doubtful whether high-speed locomotion could be controlled reliably, avoiding life-threatening situations, on the basis of judgments that were just better than chance. In steering around a bend we need to be almost certain that our trajectory is one that will keep us on the road. If a more stringent criterion of 90% correct were used in psychophysical methods then thresholds for heading judgments would be significantly higher (Wann & Land, 2000). We believe that these findings call into question the role of instantaneous heading as a control variable in locomotion. Heading can be perceived accurately during linear trajectories, but it cannot be perceived accurately during more complex trajectories. This argument also extends to Gibson s original concept of the focus of expansion (FoE), which seems to be making a re-appearance in some control models. For an observer on a curved path, or with moving gaze, the FoE for the optic flow field could be obtained from the retinal image through flow decomposition processes or a gaze subtraction process as has been proposed for instantaneous heading. But for a curved path, trajectory rotation and gaze rotation may be conflated in the retinal flow field, so 17

18 the recovery of the instantaneous translational component, and with it instantaneous FoE, may be very difficult. It is not necessary to estimate instantaneous heading or the FoE during the active steering of curved paths. The initial identification of the path when gaze is averted may be through locomotor flow lines (Lee & Lishman, 1977). Thereafter active gaze sampling of the path can enable a much finer discrimination of steering errors (Wann & Land, 2000). In line with this proposal we have observed less accurate steering when observers were required to fixate their heading, rather than their path (Wilkie & Wann, 2003). These findings do not undermine the proposals of Fajen & Warren (2003, 2004) regarding the use of the target angle β to regulate locomotion, but suggest that it is unlikely that instantaneous heading is the referent used in the specification of β. The locomotor axis (body-line of someone walking or the axis of a car or bicycle) can be used as a reference for β and the detection of the locomotor axis maybe visual or nonvisual depending upon the setting (Wann & Wilkie, 2004). The experiments presented in this paper cannot differentiate between different steering models, but our results are consistent with our previous proposals on the use of active gaze to make steering judgments. Wann & Wilkie (2004) presented an account of how observers might perceive their path and control steering on the basis of either non-visual gaze-angle information, or through the use of retinal flow. Our proposals relating to flow are that once the observer has fixated a point they wish to move towards (on a curved path) the critical component is to perceive whether ground features are then expanding along linear (but non-radial) trajectories or are curving towards the intended path. A lay-person s description would be: If you enter a bend and fixate a point in the center of the road (through which you wish to pass) then the curb and the white-line should move linearly outwards till they pass either side of your vehicle. In the case of a steering error either the curb or white line will begin to curve inwards toward your vehicle. This does not require the precise pick-up of flowfield acceleration. The results of experiment 3 seem to demonstrate that observers can respond to path information (accelerations) and we believe that skilled drivers, steering a bend, are sensitive to this information. This type of solution using active 18

19 gaze requires the perception of something equivalent to path (points you will pass through), but does not require the perception of instantaneous heading. Acknowledgements This work was supported by a grant (GR/S86358) from the United Kingdom Engineering and Physical Sciences Research Council (EPSRC, UK). Thanks to G.J.Andersen for the suggestion of the OKN experiment. For additional information including key references in pdf format, please see the authors website: 19

20 References Banks, M. S., Ehrlich, S. M., Backus, B. T. & Crowell, J. A. (1996). Estimating heading during real and simulated eye movements. Vision Res, 36, Cutting, J. E. (1986). Perception with an eye for motion. Cambridge, MA: MIT Press. Cutting, J. E., Wang, R. F., Fluckiger, M. & Baumberger, B. (1999). Human heading judgments and object-based motion information. Vision Res, 39, Fajen, B. R. & Warren, W. H. (2003). Behavioral dynamics of steering, obstacle avoidance, and route selection. J Exp Psychol Hum Percept Perform, 29, Fajen, B. R. & Warren, W. H. (2004). Visual guidance of intercepting a moving target on foot. Perception, 33, Gibson, J. J. (1958). Visually Controlled Locomotion and Visual Orientation in Animals. British Journal of Psychology, 49, Kim, N. G. & Turvey, M. T. (1999). Eye movements and a rule for perceiving direction of heading. Ecological Psychology, 11, Land, M. F. & Lee, D. N. (1994). Where we look when we steer. Nature, 369, Lappe, M., Bremmer, F. & van den Berg, A. V. (1999). Perception of self-motion from visual flow. Trends Cogn Sci, 3, Lee, D. N. & Lishman, R. (1977). Visual control of locomotion. Scand J Psychol, 18, Longuet-Higgins, H. C. & Prazdny, K. (1980). The interpretation of a moving retinal image. Proc R Soc Lond B Biol Sci, 208, Royden, C. S., Banks, M. S. & Crowell, J. A. (1992). The perception of heading during eye movements. Nature, 360, Royden, C. S., Crowell, J. A. & Banks, M. S. (1994). Estimating heading during eye movements. Vision Res, 34, Rushton, S. K., Harris, J. M., Lloyd, M. R. & Wann, J. P. (1998). Guidance of locomotion on foot uses perceived target location rather than optic flow. Curr Biol, 8, Stone, L. S. & Perrone, J. A. (1997). Human heading estimation during visually simulated curvilinear motion. Vision Research, 37, van den Berg, A. V., Beintema, J. A. & Frens, M. A. (2001). Heading and path percepts from visual flow and eye pursuit signals. Vision Res, 41, Wann, J. & Land, M. (2000). Steering with or without the flow: is the retrieval of heading necessary? Trends Cogn Sci, 4, Wann, J. P., Swapp, D. & Rushton, S. K. (2000). Heading perception and the allocation of attention. Vision Res, 40, Wann, J. P. & Swapp, D. K. (2000). Why you should look where you are going. Nat Neurosci, 3, Wann, J. P. & Wilkie, R. M. (2004). How do we control high speed steering? Optic Flow and Beyond. L. M. Vaina, S. K. Rushton and S. A. Beardsley Dordrecht, Kluwer Academic Publishers. Warren, W. H. (1998). Perception of heading is a brain in the neck. Nat Neurosci, 1, Warren, W. H., Jr. & Hannon, D. J. (1990). Eye movements and optical flow. J Opt Soc Am A, 7, Warren, W. H., Jr., Kay, B. A., Zosh, W. D., Duchon, A. P. & Sahuc, S. (2001). Optic flow is used to control human walking. Nat Neurosci, 4, Warren, W. H., Jr., Mestre, D. R., Blackwell, A. W. & Morris, M. W. (1991). Perception of circular heading from optical flow. J Exp Psychol Hum Percept Perform, 17, Warren, W. H., Jr., Morris, M. W. & Kalish, M. (1988). Perception of translational heading from optical flow. J Exp Psychol Hum Percept Perform, 14, Wilkie, R. M. & Wann, J. P. (2003). Eye-movements aid the control of locomotion. Journal of Vision, 3,

21 Figure Captions Figure 1. The display conditions for experiment 1. In condition a) gaze was stable and fixated a cross on the horizon, in b) gaze was stable, but fixation was on a static cross above the ground plane (intersecting the plane 16m from the viewer), and in c) gaze tracked a fixation cross that was attached to the ground and moved from 28.5m to 16m ahead. Figure 2. The display conditions for a) experiment 2, presenting a curved path with a constant heading and for experiment 3, presenting either b) changing path curvature, or c) rate of change of heading with a constant curvature path. Figure 3. A zoomed view (1/5 screen) of the experimental stimuli to illustrate gaze behavior over a single trial. Heading (vertical bar), path (curved line on ground) and current point of gaze (cross-hairs) superimposed, none of the superimposed features were visible to participants. Gaze fixations are superimposed in white. The numbers identify the order of gaze fixations, whilst the letters indicate alphabetically the order of saccades from fixation points. Numbers 1,3 & 5 are fixations upon the path, and 2&4 are fixations on heading. Often a second saccade was required before the observers located their fixation point. The exact moment of return of gaze was judged as the first stable fixation after the initial saccades. To analyze the stored data each trial was replayed frame by frame. Error for instantaneous heading was measured using the horizontal distance of point of gaze from the actual tangent to the curve (vertical bar). Identifying the amount of error from the path required coding rules since in theory gaze position could match any number of points on the curved path. A right-angled triangle was made connecting the PoG to two points on the path, one that was vertical from the PoG and one that was horizontal. The planar distance ( ([ h 2 + v 2 ] ) from the PoG was chosen as the corresponding point on the path,,rather than the shortest distance. This is only a reasonable approximation where the curve is at its tightest in the upper quadrant of the ground plane nearer the horizon. When the gaze dropped (due to the pull of the flow) the point on the path that was horizontally adjacent to the PoG was chosen, equivalent to the heading estimate. Figure 4. Point of gaze error when looking towards heading or a point on the linear path. Three initial fixation points were used: a static cross on the horizon (Horizon), 16m in front of the observer (16m), or a cross on the ground that moves from m (Ground). Figure 5. Point of Gaze error when looking towards instantaneous heading and at the circular path. RMS = Root Mean Squares Errors, CE = Constant Errors, deg = degrees. Figure 6. Performance on the pursuit tracking of heading or path. Panel A shows the relative angular horizontal screen displacement of heading direction and of a point on the path (16m in front of the observer) where tracking judgments were to be matched. The three final displacement values are shown, which were matched between heading and path conditions. Notice that the shape of these 21

22 functions are very similar with the path motion having a slightly steeper exponential function. Panels B and C show the performance of one participant (CH) attempting to track heading or path respectively. The solid line diagonally bisecting each plot indicates perfect performance and represents a gain function of 1.0. Note that in all cases the plots start at zero because the experimental conditions placed the cursor at the path/heading point at the start of each trial. The general pattern in panel B is one of underestimation of the heading angle, reflected in the shallow gradient and the low gain values shown in Figure 7. In panel C the gradient of path tracking performance is very close to the ideal despite being slightly offset by some early overestimation. Figure 7. Gain for the visual probe pursuit (controlled by participants) against the actual acceleration of the heading or path position shown for each individual participant. A gain of 100% would cluster around the diagonal bisector presented in Fig. 6 panels B,C. Table Caption Table 1. The root mean squared (RMS) error of gaze judgments for looking at path and heading. Accuracy of judgments are shown for the three different time periods: looking at the primary location before the tone (Before), the glance at the tone itself (Tone), then the return of gaze to the primary location (After). The arrows indicate the sequence of looking: see methods for details of the presentation conditions. 22

23 Experiment 1 Projection Screen a) b) c) FoE FoE FoE? 28.5m 16m Heading Gaze Viewer Figure 1. 23

24 a) Experiment 2 b) Experiment 3 (Condition 1) c) Experiment 3 (Condition 2) Constant Curvature Path Constant Heading Tightening Path Constant Heading Constant Curvature Path Changing Heading World Path/Heading Projection Screen Response Gaze Tracking Probe Tracking Probe Viewer Figure 2. 24

25 Figure 3. 25