Modulation Encoding in Auditory Cortex. Jonathan Z. Simon University of Maryland

Size: px

Start display at page:

Download "Modulation Encoding in Auditory Cortex. Jonathan Z. Simon University of Maryland"

Collin Weaver
5 years ago
Views:

1 Modulation Encoding in Auditory Cortex Jonathan Z. Simon University of Maryland 1

2 Acknowledgments Harsha Agashe Nick Asendorf Marisel Delagado Huan Luo Nai Ding Kai Li Sum Juanjuan Xiang Jiachen Zhuo Dan Hertz Yadong Wang Mounya Elhilali David Poeppel Shihab Shamma Jeff Walker Max Ehrman NIH: R01 AG R01 DC R01 DC R01 DC

3 Introduction Modulations what are they? Modulation Encoding What is it? Simple Examples: Modulation Transfer Functions (MTF) Complex Examples: Encoding of compound modulations, multi-am, AM & FM Effects of Attention & Hemisphere 3

4 Experimental Technique Magnetoencephalography (MEG) Awake, behaving, healthy human subjects ~150 channels (whole head) sampled at 1 khz 4

5 Auditory Modulations Amplitude Modulation (AM) Frequency Modulation (FM) Other (e.g. binaural) 5

6 Modulation Transfer Functions (MTF) Neural response for each stimulus modulation Response measured in both amplitude and phase 6

7 Steady State Response (SSR) MTF Ross et al

8 Modulation Frequency Sweep MTF: Amplitude unpublished data 8

9 Cycles Modulation Frequency Sweep MTF: Phase unpublished data 9

10 MTF Strengths & Weaknesses Strengths Linear Measure Very Powerful for Linear Systems Often Decent Characterization of modulation tuning Latency from Phase Weaknesses Linear Measure Misses known nonlinear coding strategies: fine structure response properties history dependence 10

11 Beyond the MTF: Modulation Interaction Multiple simultaneous modulation rates may interact with each other When they interact can elucidate how modulations are processed Analogy with cochlear filterbank? Modulation Filterbank? 11

12 Modulation Filterbank Evidence Cortical encoding of multiple modulations may be analogous to cochlear encoding of multiple frequencies unpublished data 12

13 Frequency Modulations FM represents different information in the stimulus than AM (e.g. formant transition) FM should therefore be processed differently than AM Single-unit studies often show FM MTFs similar to AM MTFs 13

14 Combined AM-FM Stimuli Luo et al., J. Neurophysiol (2006, 2007) 14

15 Combined AM & FM Expect direct SSR for both fam & ffm Additional encoding of slower modulations: faster SSR may itself be modulated at rate of slower SSR e.g. neural carrier at faster fam may be modulated at slower rate ffm Phase modulation? Amplitude modulation? both? 15

16 Why expect neural modulation? Ross et al. (2000) 16

17 Slowest FM rates phase modulation only Patel & Balaban (2004) 17

18 Phase Modulation only below α = (ϕupper ϕcenter) (ϕcenter ϕlower) ffm 5 Hz Luo et al., J. Neurophysiol (2006, 2007) 18

changes to Single Sideband modulation ffm ~ 5 Hz also corresponds to

19 Additional analysis shows... Transition in modulation type at ffm ~ 5 Hz Above 5 Hz, modulation type changes to Single Sideband modulation ffm ~ 5 Hz also corresponds to transition in FM psychophysics FM modulations can be encoded twice: directly, and as a secondary modulation whose coding changes at 5 Hz 19

20 The reverse (slow AM + "!((/010$2"21 Phase Modulation (PM) fast FM) is different! Amplitude Modulation (AM)! " # $ % &! &" &#!"#$%&%'(! )* (+,-. Ding & Simon, J Neurophysiol (in press) 20

21 Secondary /0( ("6'78915#3 modulation!"#$ temporally sharp %&&"'( &*% -. &*, -.!*, -. %*! -. )*+ -. /0( (";<2(3 &*:"! Ding & Simon, %&&"'( &*% -. &*, -.!*, -. %*! -. )*+ -. J Neurophysiol (in press) 21

22 Redundant Coding of Concurrent Modulations In the presence of a fast AM, a slower FM can be encoded twice: directly, and as a secondary modulation whose coding changes at 5 Hz In the presence of a fast FM, a slower AM can be encoded twice: directly, and as a secondary modulation with sharpened temporal tuning 22

23 Attentional Effects In complex auditory scenes, modulatory patterns aid segregation of auditory objects Attention to different sound components affects neural processing of the (modulated) objects 23

24 Streaming with simple modulations Elhilali et al., PLoS Biology (2009) (& more in preparation) 24

25 Attentional Enhancement Elhilali et al., PLoS Biology (2009) (& more in preparation) 25

26 Hemispheric Asymmetries Elhilali et al., PLoS Biology (2009) (& more in preparation) 26

27 Neural enhancement tracks behavior along stream Elhilali et al., PLoS Biology (2009) (& more in preparation) 27

28 Attention Modulates Modulation Encoding Attending to one stream over another enhances modulation representation Attentional modulation is hemispherically asymmetric (and task dependently so) Attentional modulation tracks behavior, over the course of the stream 28

29 Conclusions Encoding of modulations, even just as neural modulations, can be done using many different (non-linear) strategies Using stimuli with multiple modulations allows non-linear modulation encoding to shine through Modulation encoding depends on attentional state 29

Neural Coding of Multiple Stimulus Features in Auditory Cortex

Neural Coding of Multiple Stimulus Features in Auditory Cortex Jonathan Z. Simon Neuroscience and Cognitive Sciences Biology / Electrical & Computer Engineering University of Maryland, College Park Computational