Dissociating Ideomotor and Spatial Compatibility: Empirical Evidence and Connectionist Models

Transcription

1 Dissociating Ideomotor and Spatial Compatibility: Empirical Evidence and Connectionist Models Ty W. Boyer Matthias Scheutz Bennett I. Bertenthal Cognitive Science Program and Department of Psychological & Brain Sciences, Indiana University Bloomington, IN USA Abstract The tendency to imitate others is a fundamental social skill which could develop via associative learning or some more specialized mechanism, such as observation-execution matching. In this paper, we employ a stimulus-response compatibility paradigm to evaluate whether ideomotor compatibility conforms to the same processes as other S-R responses. The findings reveal a dissociation between spatial and ideomotor compatibility. A set of connectionist models are developed, which show that the differences between spatial and ideomotor compatibility are attributable to structural differences and in part to the relative strengths of an inhibitory node mediating the involuntary S-R response. Keywords: action perception; ideomotor compatibility; spatial compatibility; connectionist modeling Introduction The tendency of people to spontaneously imitate observed actions has long been noted. This automatic tendency contributes to the social glue by which humans coordinate behaviors, cooperate, and develop affiliative tendencies toward each other (Chartrand & Bargh, 1999; Lakin, Jefferis, Cheng, & Chartrand, 2003). In spite of its importance for such social interactions, there is little consensus on how this mimicry occurs. One prominent approach suggests that the perception of actions activates motor programs in an observer because each shares a common representational code (e.g., Prinz, 1990). This common coding framework descends from the ideomotor theory of James (1890) and Greenwald (1970), and has received considerable attention following the discovery of mirror neurons in the monkey brain (Rizzolatti, Fadiga, Gallese, & Fogassi, 1996). A straightforward prediction from this theory is that the ease with which a stimulus might be transformed into an action depends on the similarity or ideomotor compatibility between the observed and executed action (Brass, Bekkering, Wohlschläger, & Prinz, 2000). Imitated responses, which involve replication of observed actions, are highly ideomotor compatible. Given this formulation, it is conceivable that ideomotor compatibility is simply a special case of stimulus-response (S-R) compatibility, determined by the degree to which a stimulus and a response are perceptually, structurally, or conceptually similar (Kornblum, Hasbroucq, & Osman, 1990). Indeed, some theories explicitly propose that the same associative learning and motor control processes that underlie other S-R mapping processes also underlie imitation (Brass & Heyes, 2005; Gewirtz & Stingle, 1968). If, however, ideomotor compatibility is mediated by a neural network specialized for the direct matching of observed and executed actions (e.g., Iacoboni et al., 1999; Rizzolatti & Craighero, 2004), then it is more likely that ideomotor compatibility is dissociable from other forms of S-R compatibility. The Simon effect (Simon, 1969), a processing advantage for spatially compatible stimuli and responses, is a prime example of S-R compatibility. If ideomotor and spatial compatibility are both mediated by associative learning, then the patterns of responses on tasks that test either form of compatibility should be similar. Until recently, ideomotor and spatial compatibility have been confounded in most experiments. Bertenthal, Longo, and Kosobud (2006), however, devised a paradigm for testing both types of compatibility independently in the same experiment by manipulating the instructions. In this paradigm participants observe a tapping index or middle finger of a stimulus hand presented as if belonging to a person facing them. They respond to the tapping finger by pressing a key with the index or middle finger of their right hand. In one condition, they are instructed to imitate the cue by pressing a key with their anatomically matching finger (imitative cue). In the other condition, they are instructed to press a key with their spatially corresponding finger (spatial cue). As such, either the imitative or spatial cue is the imperative stimulus, but the other stimulus cue, although irrelevant, is also present. If the stimulus hand is the left hand (i.e., a mirror image of the participant s right hand), then the irrelevant dimension of the response is both spatially and ideomotor compatible (e.g., leftward index finger stimulus corresponds to a leftward index finger response). By contrast, if the stimulus is a right hand, then the irrelevant dimension is incompatible. Reaction times (RTs) are faster when the irrelevant dimension is compatible (i.e., left stimulus hand) than when the irrelevant dimension is incompatible (i.e., right stimulus hand). Although these studies provide evidence for both spatial and ideomotor compatibility, they also suggest that the underlying processes responsible for these two forms of compatibility are not identical. Bertenthal et al. (2006) reported that the effects of ideomotor compatibility were attenuated across a block of trials, but the effects of spatial compatibility persisted. Longo et al. (2007) also reported a 2280

2 difference, the ideomotor compatibility effect disappeared when biomechanically impossible actions were observed, although spatial compatibility continued. These findings suggest a dissociation in the processing of spatial and ideomotor compatibility, but more definitive evidence is needed. Thus, the first aim of this paper is to provide more direct evidence that spatial and ideomotor compatibility are dissociable; the second aim is to model these two processes to gain greater insight into how they differ. Experiment 1 The problem with testing for dissociation between spatial and ideomotor compatibility using the Bertenthal et al. (2006) paradigm is that similar results are predicted in both conditions. A possible solution to this problem was suggested by Sauser and Billard (2006), where ideomotor compatibility for direct and reverse S-R mapping were modeled. The logic for this manipulation is that Hedge and Marsh (1975) reported that with a reverse S-R mapping paradigm, the effect of spatial compatibility reverses (i.e., RTs are faster to stimuli that are spatially incompatible with the response). In contrast to this finding, Sauser and Billard (2006) reported model simulation results that ideomotor compatibility does not follow a similar reversal. These findings are suggestive, but they still need to be empirically tested, especially since the paradigm modeled by Sauser and Billard confounded spatial and ideomotor compatibility (cf. Bertenthal et al., 2006). In the following experiment we employ the Bertenthal et al. (2006) paradigm to compare spatial and ideomotor compatibility when participants are instructed to respond to either a direct or reverse mapping of imitative or spatial cues. If spatial and ideomotor compatibility are dissociable, then we do not expect a reverse ideomotor compatibility effect for the spatial cue condition, but do expect a reverse spatial compatibility effect for the imitative cue condition. Method Participants Forty-eight undergraduates at the University of Chicago and Indiana University (34 female; 14 male), between the ages of 18- and 25-years, participated. They were naive as to the purpose of the study, and were paid $10 or were awarded course credit for their participation. Apparatus and Materials Five-frame video sequences of a hand were displayed on a 43.2 cm computer monitor, viewed from approximately 60 cm. The hand was approximately 15 visual angle horizontally and 8 vertically. Stimulus finger movements involved a 2.5 displacement. The initial frame appeared for 533 ms and showed the hand at rest above a surface. The next three frames lasted 38 ms each and presented either the index or middle finger moving downward. The final frame lasted 886 ms and showed the finger in its final position. The stimulus was followed by a blank screen that lasted 1,467 ms. Participants responded by pressing the 1 or the 3 key on the computer keyboard number pad with their right hand index finger and right hand middle finger, respectively. E- Prime (Psychology Software Tools, Pittsburgh, PA) was used for presentation and data collection. Design and Procedure Participants were evenly assigned to one of four conditions, formed by crossing stimulus cue (imitative vs. spatial) and S-R mapping (direct vs. reversed). Direct Mapping Condition: Participants in the imitative cue condition were instructed to respond to the tapping index or middle finger with the anatomically matching finger on their right response hand. Participants in the spatial cue condition were instructed to respond to the tapping index or middle finger with the spatially congruent (left vs. right) finger on their right response hand. Reverse Mapping Condition: Participants were instructed to respond with the finger opposite that selected in the direct mapping condition. For example, they would respond to a tapping index finger in the imitative cue condition by selecting their middle finger. Participants were presented with 20 blocks 1 of 20 trials. Stimulus hand was constant within blocks and alternated between blocks (e.g., 20 trials with the left hand stimulus, then 20 trials with the right hand stimulus, and so on). The sequence was counterbalanced across subjects. With direct mapping, the left stimulus hand was compatible with the participants response hand and the right stimulus hand was incompatible. With reverse mapping, the right stimulus hand was compatible and the left hand was incompatible. Index and middle finger stimuli were presented randomly within blocks, with the constraint that each block consisted of an equal number of index and middle finger trials. Results Error trials (5.8% of all trials) and RTs less than 200 ms and greater than 1200 ms (1.9% of all trials) were excluded from all analyses. A 2 x 2 x 2 mixed model ANOVA revealed significant main effects for compatibility, F(1, 44) = 15.14, p <.001, η 2 p =.26, mapping condition, F(1, 44) = 42.44, p <.001, η 2 p =.49, and cue condition, F(1, 44) = 3.92, p = 2.05, η p =.08 (see Fig. 1), with faster responses to compatible than incompatible trials, direct than reverse mapping, and spatial cues than imitative cues. Importantly, no interactions approached significance (all ps >.18). Figure 1 illustrates the planned pairwise comparisons between compatible and incompatible trials that were statistically significant in each condition. Discussion These results replicate previous findings that have shown spatial and ideomotor compatibility effects, but were inconclusive regarding the effects of the reverse mapping. The slower RTs in the reverse mapping condition suggest that some cognitive recoding of the stimulus is necessary prior to response selection which requires additional time. In spite of this finding there was no evidence of a reversal of 1 Participants given direct S-R mapping were adopted from Bertenthal et al., 2006, and were only given 10 blocks of 20 trials. 2281

3 Figure 1: Mean RTs for compatible and incompatible trials for direct and reverse S-R mapping, imitative and spatial cues, and blocked and randomized trials. Error bars are standard errors. ** compatible-incompatible contrast at p <.05; * p <.10. the compatibility effect for either spatial or ideomotor compatibility. Our interpretation for this non-reversal is that blocking the trials had an adaptation effect on the recoding of the stimulus, and thus reduced the level of activation necessary to recode it on each trial. As a consequence of this reduced activation, the likelihood of the recoding spreading to the irrelevant priming stimulus dimension decreased. In order to address this possibility, we replicated the preceding experiment but randomized the presentation of compatible and incompatible trials within blocks. Experiment 2 Method Participants Forty-eight undergraduates at Indiana University (28 female; 20 male), between the ages of 18-and 28-years, who were not in the previous study, participated. Apparatus and Materials The same apparatus and materials of the previous experiment were used. Design and Procedure The design and procedure of the previous experiment were used; however, the stimulus hand varied randomly within each block, with the constraint that there would be an equal number of trials per block depicting a left or right hand and an index or middle finger action. Results Error trials (6.1% of all trials) and RTs less than 200 ms and greater than 1200 ms (0.5% of all trials) were excluded from the analyses. Figure 1 shows the mean RTs. A 2 x 2 x 2 mixed model ANOVA revealed significant main effects for compatibility, F(1, 44) = 6.75, p =.013, η p 2 =.13, mapping condition, F(1, 44) = 13.47, p =.001, η p 2 =.23, and cue condition, F(1, 44) = 5.26, p =.027, η p 2 =.11, with faster responses to compatible than incompatible trials, direct than reverse mapping, and spatial cues than imitative cues. Importantly, the analysis revealed significant interactions between compatibility and mapping condition, F(1, 44) = 19.15, p <.001, η p 2 =.30, mapping condition and cue condition, F(1, 44) = 7.79, p =.008, η 2 p =.15, and compatibility x mapping condition x cue condition, F(1, 44) = 4.70, p =.036, η 2 p =.10. Planned comparisons revealed faster RTs for compatible than incompatible trials in the imitative cue, direct mapping condition, t(11) = 6.11, p <.001. By contrast, RTs were faster in the incompatible than compatible trials in the imitative cue, reverse mapping condition, t(11) = -2.35, p =.038. Compatible RTs were faster than incompatible RTs in the spatial cue, direct mapping condition, t(11) = 3.32, p =.007, with no difference in the spatial cue, reverse mapping condition (p =.28). Comparisons also revealed slower RTs in the imitative cue, reverse mapping condition than each of the other conditions (all p <.006), with no differences between each of the other conditions (all p >.54). Thus, these effects were due to the compatibility reversal in the imitative cue, reverse mapping condition, but, significantly, this same reversal did not occur in the spatial cue, reverse mapping condition. Discussion The results of this experiment replicate previous findings of spatial and ideomotor compatibility effects with direct S-R mapping, and furthermore, show that the effects of spatial compatibility, but not ideomotor compatibility, reverse with reversed S-R mapping. This is in contrast with the results of the previous experiment, which suggests that randomizing the trials eliminated the adaptation effect caused by blocking the trials. As a consequence, the level of activation necessary to recode the imitative stimulus spread to the spatial priming stimulus as well. The effect of ideomotor compatibility, however, did not reverse, but rather maintained a trend toward the standard compatibility effect. This finding thus provides new and more direct evidence that priming by spatial and imitative cues differ. These results cannot tell us, however, whether this difference may be due to differences in the underlying neural pathway (e.g., architectural difference), or to differences in processing within the same pathway (e.g., parameterization difference). As a first step toward addressing this question, we develop a formal model for simulating our results. 2282

4 Computational Model and Simulations Our goal was to begin by developing the simplest possible model for simulating the results of the above experiments. Such a model will need a component representing the features of the input stimulus, which can be mapped onto a response (output units) based on matching the anatomical identity or spatial position of the input and output units. In addition, the irrelevant stimulus dimension is represented by additional S-R units. Figure 2: The base model (upper left) and models for the direct (upper right) and reverse (bottom) mapping conditions (arrows indicate excitatory connections, circles inhibitory connections, see text for explanations of labels and weights). Model Architecture and Parameters We use interactive activation and competition connectionist units whose change in activation over time is given by act/ t = netin - act (netin + decay), where act [0,1] is the activation of the unit, netin [0,1] the sum of the weighted inputs to the unit and decay [0,1] is a constant decay factor (set to 0.05 for all nodes). The sum of all incoming connection weights must be at most 1 (to guarantee that netin [0,1]). The base model consists of six units (upper left in Fig. 2): two input units, called finger units, representing the perceived index ( I ) vs. middle ( M ) input finger, two input units, called location units, representing the left ( L ) vs. right ( R ) location of the perceived input finger (depending on the stimulus hand) as well as two output units, representing the index finger in the left location ( IL ) vs. the middle finger in the right location( MR ), corresponding to the right hand of the participant. Input units are connected to output units via direct excitatory connections: left location and index finger inputs to the left index finger output; right location and middle finger inputs to the right middle finger output (the strength of all connections is fixed at 0.001). Inputs are applied to the model by adding a fixed external activation of 0.5 to the netinput of the respective input units on each cycle. A response is selected whenever an output unit first reaches the action threshold of 0.5 (i.e., the activation needed to perform a motor action of the finger). The weights in the base model reflect task-independent connections present in participants before, and lasting beyond, the task, and these connections are insufficient to create an output action even if all input units are activated. Parameter Fitting We constructed eight models for the 2x2x2 design: compatibility (spatial vs. ideomotor), condition (direct vs. reverse mapping), and presentation (blocked vs. randomized). Model parameters were obtained by fitting the grand means of the human data from the first ten blocks of the experiment (See Fig. 3). The models were then incrementally constructed from the base model: first, we add mutually inhibitory connections to the two output fingers because the task requires subjects to move only one finger (and not the other) at any given time (the weights are fixed at for all eight models). For the direct mapping conditions (see upper right in Fig. 2), we added additional S-R mapping units ( SR ) to the base model, with excitatory connections coming from either the finger or the location input units, depending on the stimulus condition, with connections going to the output units: for the imitative cue (i.e., spatial compatibility), input fingers (i.e., index and middle) are connected to the SR units, which are then connected to matching output units; for the spatial cue (i.e., ideomotor compatibility), input locations (i.e., left and right) are connected to SR units, which are then connected to matching output units. We then fit the additional excitatory connections to the human data. The best fitting connection values are 0.86 for the random models and 0.1 for the blocked models. The difference reflects the adaptation effect in the blocked conditions, which emerged because the stimulus hand was the same for the entire block, allowing a consistent input finger type/location and response finger type/location mapping. In the second experiment, the stimulus hand was randomized from trial to trial, thus preventing the learning of a fixed mapping within the block. Note, the SR units and their excitatory connections and the inhibitory connections between output units represent temporary connections, all based on the task instructions (assumed to not exist in participants before the experiments). For the reversed mapping conditions (bottom in Fig. 2), we extend the direct mapping models by adding two reversal units ( Rv ) whose excitatory connections parallel those of the SR units (i.e., connecting input finger type or location unit to the corresponding output unit). Moreover, we add inhibitory connections between the reversal units and the corresponding SR units, because the direct mapping established by the SR units has to be explicitly suppressed. We can then fit both the added excitatory and inhibitory connections to the human data. The best fitting excitatory weights are 0.92 for all random and 0.1 for all blocked models. The inhibitory weights reveal an important difference between spatial and ideomotor models: the best fitting inhibitory weights are lower for the spatial than for the ideomotor models in both 2283

5 Figure 3: Parameters used for fitting the models and model results. Parameters represent the grand mean of the human data in the first ten blocks of trials. LH = Left hand; RH = Right hand. random and blocked conditions. In the random condition, the inhibitory weights are for the spatial and for the ideomotor condition. In the blocked condition, the inhibitory weights are -0.1 for the spatial condition, and again for the ideomotor condition. The lower magnitudes on the connections in the blocked condition are again due to within-block adaptation. Simulation Results and Discussion The simulation results (Fig. 3) show a very good fit of the model to all experimental conditions, except the randomized, reverse mapping, imitative cue (spatial compatibility) condition. This suggests that the models are able to capture the essential results of the tasks other than the reversed spatial compatibility effect. That the model was unable to fit the human data in this particular condition suggests that additional nodes, not present in our current models, may be necessary to explain performance in this condition. It is also interesting that the model fit the complimentary spatial cue (ideomotor compatibility) condition, suggesting that no additional components may be necessary in this condition. Another key result was the difference in inhibitory connections across models, which hints at an important difference between the way humans process ideomotor and spatial compatibility. While our models are silent on many processing details (i.e., stimulus encodings and operations), they do capture important differences in processing time and effort. For the reversed mapping conditions, the consistently larger inhibitory connections in the ideomotor than in the spatial compatibility models indicate greater suppression underlying ideomotor compatibility. The reversed models thus demonstrate that the locus for the processing difference between spatial and ideomotor compatibility may involve the inhibitory connections between the S-R and reversal units, because no other connection in the random and blocked reversed models differ. General Discussion Two experiments tested for dissociation of ideomotor and spatial compatibility and a set of computational models were developed to identify potential differences between these two processes. The first experiment, where compatibility varied only between blocks, showed that the effects of neither spatial nor ideomotor compatibility reversed with a reverse S-R mapping paradigm. The second experiment, where compatibility varied randomly within blocks, showed that the effect of spatial compatibility reversed with reverse mapping, but the effect of ideomotor compatibility did not. This dissociation is consistent with the suggestion that ideomotor compatibility is mediated by a neural network specialized for the direct matching of observed and executed actions (e.g., Iacoboni et al., 1999; Rizzolatti & Craighero, 2004). 2284

6 Our connectionist models allow us to draw further inferences about the processes that underlie ideomotor and spatial compatibility. In the reverse mapping conditions, both ideomotor and spatial compatibility are activated via additional reversal units with inhibitory links to the SR nodes, but the model was unable to fit the spatial compatibility condition, suggesting additional nodes are necessary to fully explain spatial compatibility reversal effects. Importantly, a difference that did emerge between spatial and ideomotor compatibility was in the inhibitory weights of these nodes. The greater inhibition associated with ideomotor compatibility may reflect the early origins and encapsulated stability of the neural circuit responsible for the direct matching of perceived actions by the observer s motor system (Rizzolatti & Craighero, 2004). The greater inhibition associated with ideomotor compatibility also makes sense in terms of the processes underlying social engagement. For example, social cognitive research has shown that although spontaneous mimicry can be socially advantageous (e.g., Lakin et al., 2003), excessive mimicry of others can be socially undesirable, and has long been known as a pathological manifestation of frontal lobe brain injury (Stengel, 1947). Conclusion and Future Work In this paper we provided new experimental evidence that suggests ideomotor and spatial compatibility are dissociable, and therefore are not likely attributable to the same underlying associative system. We also presented a set of computational models fit to the data for explaining the dissociation of spatial and ideomotor processes. The models suggest structural differences as well as different inhibitory connections that may be required to suppress direct SR mappings between observed and executed actions. This difference in inhibitory connections could be related to a more complex model where the cognitive reversal rule (implemented in the Rv structure in the current model) spreads more strongly to the priming stimulus in the spatial compatibility models. This spreading of the cognitive rule suggests that a slower semantic process replaces and reverses the more direct S-R mapping mediated by spatial compatibility. Developing this model further and extending it to explain the differences between blocked and randomized trial compatibility effects is left for future work. References Bertenthal, B. I., Longo, M. R., & Kosobud, A. (2006). Imitative response tendencies following observation of intransitive actions. Journal of Experimental Psychology: Human Perception & Performance, 32, Brass, M., Bekkering, H., Wohlschläger, A., & Prinz, W. (2000). Compatibility between observed and executed finger movements: Comparing symbolic, spatial, and imitative cues. Brain & Cognition, 44, Brass, M. & Heyes, C. (2005). Imitation: Is cognitive neuroscience solving the correspondence problem? Trends in the Cognitive Sciences, 9, Chartrand, T. L., & Bargh, J. A. (1999). The chameleon effect: The perception behavior link and social interaction. Journal of Personality and Social Psychology, 76, Gewirtz, J. L. & Stingle, K. G. (1968). Learning of generalized imitation as the basis for identification. Psychological Review, 75, Greenwald, A. G. (1970). Sensory feedback mechanisms in performance control: With special reference to the ideomotor mechanism. Psychological Review, 77, Hedge, A., & Marsh, N. W. A. (1975). The effect of irrelevant spatial correspondence on two-choice responsetime. Acta Psychologica, 39, Iacoboni, M., Woods, R.P., Brass, M., Bekkering, H., Mazziotta, J.C., & Rizzolatti, G. (1999). Cortical mechanisms of human imitation. Science, 286, James, W. (1890). The Principles of Psychololgy, vol 2. Henry Holt & Co: New York, NY, US. Kornblum, S., Hasbroucq, T., & Osman, A. (1990). Dimensional overlap: Cognitive basis for stimulusresponse compatibility a model and taxonomy. Psychological Review, 97, Lakin, J. L., Jefferis, V. E., Cheng, C. M., & Chartrand, T. L. (2003). The chameleon effect as social glue: Evidence for the evolutionary significance of nonconscious mimicry. Journal of Nonverbal Behavior, 27, Longo, M.R., Kosobud, A., & Bertenthal, B.I. (2007). Automatic imitation of biomechanically possible and impossible movements: Effects of priming movements vs. goals. Journal of Experimental Psychology: Human Perception & Performance, 34, Prinz, W. (1990). A common coding approach to perception and action. In O. Neumann & W. Prinz (Eds.), Relationships between perception and action (pp ). Berlin: Springer-Verlag. Rizzolatti, G. & Craighero, L. (2004). The mirror-neuron system. Annual Review of Neuroscience, 27, Rizzolatti, G., Fadiga, L., Gallese, V., & Fogassi, L. (1996). Premotor cortex and the recognition of motor actions. Cognitive Brain Research, 3, Sauser, E. L., & Billard, A. G. (2006). Parallel and distributed neural models of the ideomotor principle: An investigation of imitative cortical pathways. Neural Networks, 19, Simon, J. R. (1969). Reactions toward the source of stimulation. Journal of Experimental Psychology, 81, Stengel, E. (1947). A clinical and psychological study of echo-reactions. Journal of Mental Science, 93,