The Data: Multi-cell Recordings

Transcription

1 The Data: Multi-cell Recordings What is real? How do you define real? If you re talking about your senses, what you feel, taste, smell, or see, then all you re talking about are electrical signals interpreted by your brain. Morpheus, The Matrix In this course everything will be presented in the context of real neural data of one form or another. Working with real data builds intuitions for the problem that are impossible to come by otherwise. The challenge is that real data can sometimes be a mess; data can be missing, corrupted by noise, or not be in the format you want. Despite this, we will use the real data to introduce concepts, and ground them in reality, throughout the course. We will work with two basic neural datasets. ffl a center-out reaching task [3] and ffl a continuous tracking task [8]. We are very fortunate to be able to work with real data gathered at Brown by one of the best groups in the world. The kind of data we have to work with is available to few others and is graciously provided by Nicho Hatsopoulos and John Donoghue from the Department of Neuroscience. The data is stored in /course/cs295-7/data/ You have read permission for this data. I ask that you do not copy it elsewhere, but rather, load it from this directory. This is current research data and is hence to be protected; do not distribute it and do not copy it to an unprotected directory. 1

2 2 CS295-7 cfl Michael J. Black, 2001 Chosing a particular dataset does have some limitations; not all parts of the brain have the same function and spatial organization. The hope is that the tools we use to understand this particular dataset will be applicable to other parts of the brain and other problems. B Central Sulcus MI SMA PMA PMA Arcua te 5mm Figure 1: Brain of a macaque monkey with implant shown in the arm area of primary motor cortex (MI). Motor Cortex We will be working with data from the arm area of motor cortex. There is a good deal of evidence to suggest that cells in this area encode information about the position and velocity of the hand [2, 3]. In our decoding view of the problem this means that by understanding the collective behavior of the cells in this region, we should be able to infer the motion of the animal s hand. We will take this as our goal. This may seem surprising. Why should an individual cell in the arm area have anything to do with hand position and velocity? One might, for example, expect the cells to encode information about the joint angles in the arm, the muscle forces required to move it, or the torque on the joints. Each of these can be thought of as different representations for describing the motion of a hand over time. Since each of these descriptors (joint angle, muscle acitivation, hand velocity, etc) can tell us something about how the hand is moving, they are highly correlated. In the language of the next Section, this means that joint angles and hand positions are not statistically independent. In other words, knowing something about hand position tells us something about arm joint angles (this is formalized as inverse kinematics in robotics) and vice versa. Consequently, when we look at the neural data, we may hypothesize that the cells are encoding some property of the world and, if this property is somehow correlated with hand position, we are likely to find evidence for it. We will need to be careful about making hypotheses and then infering too much from them. We ll talk more about this later.

3 Brain Computer Interfaces: Computational and Mathematical Foundations 3 Figure 2: Cochlear implant. Left: linear array of electrodes follows the interior contour of the cochlea and stimulates different spatial locations along its length. These locations correspond to different temporal frequencies of the incoming sound wave. Right: Commerical implant technology is available from a number of companises (image from AllHear). The cells we record from in primary motor cortex lack a strict somatopy;that is, their spatial organization does not seem to encode information. While different regions of MI do encode the motion of different parts of the body, within the arm area the cells all appear to be doing similar jobs. This is in contrast to other areas of interest for BCI. For example, visual prostheses take advantage of the fact that primary visual cortex (area VI) is retinotopically mapped. In this area, the spatial position of cells in the brain corresponds to the spatial position of receptors in the eye and, thus, stimulating neighboring cells in VI results in the perception of neighboring stimuli. Similarly, the cochlea in the inner ear is tonotopically mapped. Its shape is such that it vibrates at different locations along its length depending on the temporal frequencies of the sound signal. Neurons leading from the cochlea to the brain then encode different frequencies and this spatial origanization is used in the design of cochlear implants. These implants typically consist of a linear array of electrodes that are implanted into the cochlea and which stimulate different parts of its length depending on the temporal frequencies of the incoming sound waves. Recording A ten by ten array of electrodes is implanted in the primary motor cortex (MI) of a Macaque monkey (Figures 3 and 4). These arrays have produced high-quality recordings of individual neurons in awake, behaving, monkeys for up to three years. The exact neurons recorded by each electrode may change from day to day

4 4 CS295-7 cfl Michael J. Black, 2001 A Figure 3: Implantable electrode array. Left: Utah Intracortical Electrode Array from Bionic Technologies Ltd. Right: Illustration of a single electrode in neural tissue. (Figures from Bionic Technologies Ltd.) if the array position shifts slightly (this can be caused by the tethering forces of the external wire bundle. A fully implantable device is being developed which should minimize this shifting. The consequence of this shifting for BCI research is that one must recalibrate the system to detect and account for such changes. For various technical reasons we do not have recordings from all 100 cells. Currently we have good recordings from approximately 25 cells and we take this as the input data to be analyzed. A variety of information can be detected and recorded from the electrodes. For example, local field potential recordings provide information about the electrical activity of a large group of cells in the surrounding tissue. This could be very useful information but in this course we will focus on the activity of individual neurons. If we look at the electrical activity in the temporal frequency range of individual neural firing (on the order of a millesecond (ms)), we see repeated waveforms such as those in Figure 5. Since the shape of this waveform is highly repeatable, this specific shape is not likely to carry information. Hence we can think of the occurance of the waveformas an atomic event, or spike. Software will automatically detect these spikes and their exact timing; this will be the data we work with. 1 To learn what the cells encode, we must have examples of their behavior (spiking) in different situations. We will consider two basic tasks below and simulataneously record the neural activity and the behavior of the animal during the task. From this, we will later be able to construct a model of what the cells encode. 1 Sometimes an electrode is close to two cells and is picking up the activity of both of them. There exists automatic spike sorting software that analyzes the waveforms and gives the spiking behavior of both cells. We will treat this as a black box but should realize that the algorithms that do this sorting may not be perfect and this could be a source of error, or noise, in our data.

5 Brain Computer Interfaces: Computational and Mathematical Foundations 5 Acrylic Silicone Connector Bone Dura I 500 µm White Matter 400 µm Cortex III V VI Figure 4: Diagram showing implanted array in cortex. The dura is closed over the top of the array. Ordinarily a sheet of Teflon is placed between the dura and the array back and a sheet of Gortex is placed above the dura. The defect is covered with silicone and finally the wire bundle and silicon are covered with cranioplast cement. (Figure curtesy of N. Hatsopoulos.) C 80µV 1ms Figure 5: Superimposed waveforms from the spiking activity of a single neuron. (Figure curtesy of N. Hatsopoulos.) Center-Out Task A variety of behavioral tasks are possible. The most common task in the literature involves reaching in one of a fixed number of directions [2, 3, 10]. The monkey moves their arm on a 2D tablet while holding a low-friction manipulandum (Figure 6). A trial consists of a number of phases: 1. recording starts 2. the monkey receives a visual cue indicating in which of the eight directions they will move 3. they are given go cue 4. after the cue, they move as quickly as possible to the target 5. the recording ends.

6 6 CS295-7 cfl Michael J. Black, 2001 Possible targets Movement target Position Feedback Cursor Center hold VideoMonitor Digitizing Tablet Digitizing Tablet A B C Figure 6: Center-out task. The monkey reaches in one of eight directions (from N. Hatsopoulos). In homework Assignment 2, you will have data in this form. The task is further illustrated in Figure 7. The motion is repeated many times and the exact path of the hand motion varies slightly. For this task we do not know the exact hand path but only which target has been cued. Consequently, with this data, we will be able to detect which direction the monkey moves but not the exact path of that motion. Figure 8 illustrates how the neurons behave during this task. Note that this cell has a preferred direction of motion; that is, a direction for which it is most active. Also note that the motion 180 degrees opposite is least preferred in the sense that the firing is lowest. There is a predictable variation in the cell s response as a function of movement direction,. The cell s mean firing can be modeled by f 1=2 = b 0 + b x sin( )+b y cos( ) where b 0, b x,andb y are constants. This is referred to as cosine tuning [7] and we will talk more about it later. The center-out task involves balistic motion in the case where the subject knows the movement direction ahead of time. This corresponds roughly to the scenarios such as reaching or pushing a buttom. These are useful tasks to model for BCI applications. For example, we might like to build a robot arm that can be controled to do these common tasks. Below we consider another type of motion. Continuous Tracking Task The center-out task does not tell us directly about how the cells in motor cortex other types of tasks such as the smooth motion of a hand in space. To explore this we will consider a more complex scenario in which the monkey performs a

7 Brain Computer Interfaces: Computational and Mathematical Foundations 7 A B 40 Speed (cm/s) Time (ms) Figure 7: Center-out task. The subject moves as quickly as possible from the central starting position to one of eight targets. There is variablility in their motion as illustrated on the left. The right figure shows the speed of their motion. The shape of this curve is typical of ballistic motions. (From Moran and Schwartz 99a). continuous tracking task [8]. The hand position controls the position of a feedback dot presented on a computer screen (Figure 9). The monkey s task is to manually follow a target on the screen by moving its hand. A successful trial requires that the feedback dot remain within a pre-specified distance of the target. At the end of a successful trial the monkey receives a juice reward. The path of the target is chosen to be a smooth random walk that effectively samples the space of hand positions and velocities (Figure 10) [8]. Target positions over all trials can be represented by a two-dimensional Gaussian process. Similarly, velocities have a roughly Gaussian distribution over speeds, r, with a peak at zero. Directions of the target motion were uniformly distributed (though clear peaks corresponding to particular directions in Figure 10 (b) indicate that the monkey does not track the target in an unbiased manner). In this task we do not have the same burst of activity that characterizes the beginning of the center-out task. For more details see /course/cs295-7/assignment1/assign1.m That file has example code for plotting the x; y trajectories and visualizing the spiking data.... more details to come...

8 8 CS295-7 cfl Michael J. Black, Figure 8: Neural acitivity, center-out task. The response of a single cell to different movement directions is shown. Left: For a particular direction, the figure shows rows corresponding to firing activity of the cell over the time of the trial. We can see that there is an intial burst of activity at the beginning of the task and this gradually drops off. With each row corrsponding to a different trial, we see that the exact pattern of activity varies but is statistically similar. Right: the data from the multiple trials are averaged together and plotted as a histogram of firing. (From Moran and Schwartz 99a). Monitor Target Tablet Trajectory Manipulandum Figure 9: Continuous tracking task. The subject moves the manipulandum on a 2D surface so as to track a target on the screen.

9 Brain Computer Interfaces: Computational and Mathematical Foundations a 0 b 0 Figure 10: Distribution of the position (a) and velocity (b) of the monkey s hand during tracking tasks. The target moves with a smooth random walk. Color coding indicates the frequency with which different parts of the space are visited. (a) Position: horizontal and vertical axes represent x and y position of the hand. (b) Velocity: the horizontal axis represents direction, ß» <ß, and the vertical axis represents speed, r.

10 10 CS295-7 cfl Michael J. Black, 2001

11 Bibliography [1] J. P. Donoghue, J. N. Sanes, N. G. Hatsopoulos, and G. Gaal. Neural discharge and local field potential ocscillations in primate motor cortex during voluntary movements. Journal of Neurophysiology, 79: , [2] Q-G. Fu, D. Flament, J.D. Coltz, and T.J. Ebner. Temporal encoding of movement kinematics in the discharge of primate primary motor and premotor neurons. J. of Neurophysiology, 73(2): , [3] A. P. Georgopoulos, A. B. Schwartz, and R. E. Kettner. Neuronal population coding of movement direction. Science, 233: , [4] N. G. Hatsopoulos, C. L. Ojakangas, L. P. Paninski, and J. P. Donoghue. Information about movement direction obtained from synchronous activity of motor cortical neurons. Proc. National Academy of Sciences, 95: , December [5] Maynard, E.M., C.T. Nordhausen, and R.A. Normann, The Utah Intracortical Electrode Array: a recording structure for potential brain-computer interfaces. Electroencephalogr. Clin. Neurophysiol., (3): p [6] E. M. Maynard, N. G. Hatsopoulos, C. L. Ojakangas, B. D. Acuna, J. N. Sanes, R. A. Normann, and J. P. Donoghue. Neuronal interaction improve cortical population coding of movement direction. Journal of Neuroscience, 19(18): , September [7] Moran and Schwartz. [8] L. Paninski, M. R. Fellows, N. G. Hatsopoulos, and J. P. Donoghue. Temporal tuning properties for hand position and velocity in motor cortical neurons. in preparation, [9] F. Rieke, D. Warland, R. de Ruyter van Steveninck, and W. Bialek, editors. Spikes: Exploring the Neural Code. MIT Press, Cambridge, MA,

12 12 CS295-7 cfl Michael J. Black, 2001 [10] J. Wessberg, C. R. Stambaugh, J. D. Kralik, P. D. Beck, M. Laubach, J. K. Chapin,J.Kim,S.J.Biggs,M.A.Srinivasan,andM.A.L.Nicolelis.Realtime prediction of hand trajectory by ensembles of cortical neurons in primates. Nature, 408: , November 2000.