Learned Integration of Visual, Vestibular, and Motor Cues in Multiple Brain Regions Computes Head Direction During Visually Guided Navigation

Transcription

1 HIPPOCAMPUS 22: (2012) Learned Integraton of Vsual, Vestbular, and Motor Cues n Multple Bran Regons Computes Head Drecton Durng Vsually Guded Navgaton Bret Fortenberry, Anatol Gorchetchnkov, and Stephen Grossberg* ABSTRACT: Effectve navgaton depends upon relable estmates of head drecton (HD). Vsual, vestbular, and outflow motor sgnals combne for ths purpose n a bran system that ncludes dorsal tegmental nucleus, lateral mammllary nucle, anteror dorsal thalamc nucleus, and the postsubculum. Learnng s needed to combne such dfferent cues to provde relable estmates of HD. A neural model s developed to explan how these three types of sgnals combne adaptvely wthn the above bran regons to generate a consstent and relable HD estmate, n both lght and darkness, whch explans the followng expermental facts. Each HD cell s tuned to a preferred head drecton. The cell s frng rate s maxmal at the preferred drecton and decreases as the head turns from the preferred drecton. The HD estmate s controlled by the vestbular system when vsual cues are not avalable. A well-establshed vsual cue anchors the cell s preferred drecton when the cue s n the anmal s feld of vew. Dstal vsual cues are more effectve than proxmal cues for anchorng the preferred drecton. The ntroducton of novel cues n ether a novel or famlar envronment can gan control over a cell s preferred drecton wthn mnutes. Turnng out the lghts or removng all famlar cues does not change the cell s frng actvty, but t may accumulate a drft n the cell s preferred drecton. The antcpated tme nterval (ATI) of the HD estmate s greater n early processng stages of the HD system than at later stages. The model contrbutes to an emergng unfed neural model of how multple processng stages n spatal navgaton, ncludng postsubculum head drecton cells, entorhnal grd cells, and hppocampal place cells, are calbrated through learnng n response to multple types of sgnals as an anmal navgates n the world. VC 2012 Wley Perodcals, Inc. KEY WORDS: head drecton cells; learnng; postsubculum; vestbular sgnals; outflow motor sgnals; dstal vsual cues; spatal navgaton; vsual learnng; motor-vestbular calbraton INTRODUCTION Spatal navgaton s a crtcal means to acheve behavoral success n many terrestral anmals. Rats use t to effcently fnd food and quckly Center for Adaptve Systems, Department of Cogntve and Neural Systems, and Center of Excellence for Learnng n Educaton, Boston Unversty, Boston, Massachusetts Grant sponsor: CELEST, an NSF Scence of Learnng Center; Grant number: SBE ; Grant sponsor: SyNAPSE program of the Defense Advanced Research Projects Agency; Grant number: HR C *Correspondence to: Stephen Grossberg, Center for Adaptve Systems, Department of Cogntve and Neural Systems And Center of Excellence for Learnng n Educaton, Scence and Technology Boston Unversty, 677 Beacon St, Boston, MA E-mal: steve@bu.edu Accepted for publcaton 16 Aprl 2012 DOI /hpo Publshed onlne 18 June 2012 n Wley Onlne Lbrary (wleyonlnelbrary.com). return to a safe locaton. Ths task requres that a rat contnuously update ts current poston and the drecton towards the safe poston. A key step n ths process s to estmate an anmal s drecton n the envronment along the entre path to the safe poston. Small errors n both self-localzaton and drecton could accumulate over dstance traveled, and thereby lead the rat drastcally off course. Sngle and multple cell recordngs n rats have produced detaled nformaton about cells that underle both processes: Head Drecton (HD) cells mantan drectonal estmates, whle entorhnal grd cells and hppocampal place cells code an anmal s poston. HD cells are found n the lmbc system (Ranck, 1984; Blar and Sharp, 1995; Taube, 1995; Redsh et al., 1996; Stackman and Taube, 1997) and utlze nputs from the vestbular (Blar and Sharp, 1996; Stackman and Taube, 1997; Goodrdge et al., 1998), motor (Taube, 1998, 2007), and vsual (Taube, 1995; Zugaro et al., 2001, 2003) systems to produce and mantan a sgnal that correlates to the horzontal drecton of the head relatve to the body axs. The frng rate of a HD cell s maxmal at the cell s preferred drecton and decreases as the head turns away from the preferred drecton. There s a unform dstrbuton of preferred drectons among the populaton of HD cells (Taube, 1998). The vestbular system s the prmary source of the HD sgnal. If the vestbular system s removed, propertes of the HD cells are lost (Stackman and Taube, 1997; Taube, 1998). Although the vestbular system s crtcal for generatng a HD cell sgnal, the sgnal s also nfluenced by nputs from outsde the vestbular system. Motor nfluences are revealed by experments that study the effects of passve versus actve rotatons (Knerm et al., 1995; Taube, 1995; Zugaro et al., 2001; Taube and Bassett, 2003). A vsual landmark can correct the drecton that has been mantaned by the vestbular and motor systems (Taube, 1995; Zugaro et al., 2001, 2003, 2004; Yoganarasmha et al., 2006). The actvty of HD cells antcpates the drecton that s assumed by the rat s head durng head rotatons. The tmng of ths antcpaton s referred to as the antcpatory tme nterval (ATI), and s observed n the lateral mammllary nucle and anteror dorsal thalamc nucle. It has been theorzed that the antc- VC 2012 WILEY PERIODICALS, INC.

2 2220 FORTENBERRY ET AL. FIGURE 1. Tme slde analyss on a populaton of angular head velocty (AHV) cells n the left hemsphere. The actvty below baselne occurs durng perods precedng a head rotaton and the actvty above baselne occurs durng perods succeedng a head rotaton. The psversve hemsphere (counter clockwse) antcpates the head rotaton whle the contraversve hemsphere (clockwse) s delayed. The HeadMoVVes model assumes that the dfferences are due to motor and vestbular AHV nputs, respectvely. [Reprnted wth permsson from Sharp (2001)] paton s due to motor and vestbular system nteractons (Blar and Sharp, 1995; Stackman and Taube, 1997). Indeed, the ATI s greater durng passve rotatons than actve rotatons (Bassett et al., 2005). Rate-based and spkng models have been developed to explan how HD cell responses can be generated and mantaned usng vestbular angular head velocty nputs (Blar and Sharp, 1995, 1996; Redsh et al., 1996; Goodrdge and Touretzky, 2000; Boucheny et al., 2005; Song and Wang, 2005). These models have demonstrated accurate and robust HD responses durng head rotatons. Some of the models also suggested how vsual cues may nfluence the HD sgnal. All these models used smple structures wth only a few of the regons known to contan HD cells. They have not addressed how these crcuts adaptvely calbrate vestbular, motor, and vsual sgnals to generate consstent commands n the multstage bran crcuts that carry out HD computaton, how dfferent tmng rates occur n the two bran hemspheres durng a head rotaton, and how motor nputs can contrbute to the ATI effect. In ths paper, a neural model, called the HeadMoVVes (Head drecton from Motor, Vsual, and Vestbular sgnals) model, s developed to explan how vestbular, motor, and vsual cues combne through learned nteractons to generate a consstent and relable HD estmate, under lght or dark condtons, and to explan the followng types of data: how motor and vestbular system nteractons can produce an ATI shft, why ATI shfts dffer for clockwse and counter-clockwse head rotatons, why so many regons n the lmbc system contan HD cells, why smpler crcutry cannot accomplsh these tasks, how vsual landmarks reset the HD cell s preferred drecton, and why dstal landmarks have a stronger effect on the frng propertes of HD cells than proxmal landmarks. These results have been brefly reported n Fortenberry et al. (2009a,b). MATERIALS AND METHODS Ths secton summarzes expermental data about the propertes of head drecton (HD) cells, prevous rng attractor models of HD cells, and the HeadMoVVes rng attractor model that s used to explan and smulate key expermental data. Expermental Propertes of Head Drecton Cells Drectonal tunng HD cells are characterzed by ther tunng curve durng a 3608 rotaton. Cell frng s maxmal at the preferred drecton and decreases as the head rotates n ether drecton away from the preferred drecton (Fg. 1). The tunng curve can ether have a trangular, cosne or Gaussan shape (Blar and Sharp, 1995; Taube, 1995). The cell s defned by the preferred frng drecton, peak frng rate, tunng curve wh, and baselne frng, whch s at or near zero for most cells (Taube, 1998). The tunng curve wh vares wth bran regon from 608 to 1508 at base rate. The peak-frng rate also depends on bran regon and can range from 16 (spkes/s) to 120 (spkes/s) (Taube and Bassett, 2003). The cell s preferred drecton s nfluenced by the vestbular, motor, and vsual systems. When vsual nput s not avalable, the vestbular and motor systems sustan HD actvty but may accumulate a slow drft of the preferred drecton (Knerm et al., 1998). However, a well-establshed vsual cue can reset the preferred drecton and consstently anchor t to a landmark (Taube, 1995; Zugaro et al., 2001, 2003, 2004; Yoganarasmha et al., 2006). Hemspherc dfferences n HD cells The HD system n rats receves angular head velocty (AHV) nput from the nucleus prepostus hypogloss (PPH) and the

3 ADAPTIVE CALIBRATION OF HEAD DIRECTION 2221 medal vestbular nucleus (MVN), whch project to the dorsal tegmental nucleus (DTN). The DTN contans both AHV cells and AHV-HD combnaton cells (Stackman and Taube, 1997; Blar 1998; Taube, 1998). Sharp et al. (2001) analyzed the tmng of AHV cell actvaton n the DTN n the two hemspheres n response to head rotatons n both the clockwse and counter-clockwse drectons. Ther tme slde analyss correlates the frng rate of the AHV cells to the rat s head rotaton for recent past, present, and near future. Cell actvty s measured n 16.7-ms ntervals from 1,000 ms pror to a head rotaton to 1,000 ms after a head rotaton. Fgure 1 shows the result of averagng a populaton of cells n the left hemsphere for both clockwse and counter-clockwse tme slde correlates across four consecutve samples of head rotatons n the left hemspheres wth a mnmum turnng rate of 250 deg/s. The two curves represent two dfferent drectons of head movements. Durng a clockwse rotaton, the left hemsphere s the contraversve hemsphere. Durng a counter-clockwse rotaton, the left hemsphere s the psversve hemsphere (Fg. 1). The AHV sgnal n the contraversve hemsphere durng a clockwse rotaton lags untl after the head rotaton. The AHV sgnal n the psversve hemsphere durng a counter-clockwse rotaton peaks pror to the head rotaton. The HeadMoVVes model assumes that AHV cell actvaton obeys the same laws for clockwse and counter-clockwse head movements n both hemspheres of the DTN. The data for the rght hemsphere n Sharp et al. (2001, Fg. 7, mddle panel) show a smlar qualtatve effect as n the left hemsphere, but the counterclockwse peak s attenuated. Moreover, ths trend contnues to be seen, although weaker than for the left hemsphere alone, n the combned hemsphere data (Fg. 7, bottom panel). Currently, the reason for ths asymmetrc actvaton s unclear. Addtonal data would be welcome to clarfy ts bass. For smplcty, the model currently assumes symmetrc actvaton f only because clockwse and counterclockwse rotatons of the head both occur, and the most parsmonous desgn s one that would regulate them n a smlar way across hemspheres. Two types of nputs: Vestbular and motor outflow The types of cells found n the DTN can explan the tmng effects found n the tme slde analyss. The PPH of the vestbular system contans two types of nputs based on dfferent har cells n the semcrcular canal. Type I cells respond to psversve head rotatons and Type II cells respond to contraversve head rotatons (Gdowsk and McCrea, 2000; Klam and Graf, 2003). The PPH contans about twce as many Type II than Type I vestbular neurons (Blar et al., 1998; Lannnou et al., 1984). The DTN, n turn, receves vestbular nformaton from the PPH. Thus, t s reasonable to assume that delayed AHV sgnals to the contraversve hemsphere of the DTN are drven by Type II vestbular neurons, whle antcpatory sgnals to the DTN psversve hemsphere are drven by another source. The HeadMoVVes model uses motor outflow n the psversve hemsphere and does not address the functonal role that Type I vestbular neurons may have. Ths source s heren assumed to be motor outflow, or corollary dscharge, sgnals. The nterpeduncular nucleus (IPN) has been mplcated as a possble source of motor outflow sgnals to the DTN (Sharp et al., 2006; Taube et al., 2009). The IPN has recprocal connectons to the DTN and has neurons senstve to movement speed. Lesons of the IPN (Taube et al., 2009) attenuate the preferred drecton frng rate n ADN HD neurons. Passve head rotatons wthout motor sgnals have a less clear effect on cell frng rates. Some older studes (Knerm et al., 1995; Taube et al., 1990; 1995) suggested that the frng rates are attenuated, whle more recent data by Shnder and Taube (2011) suggest that there s no attenuaton of frng rate n the case of passve rotatons. Regon-specfc propertes of HD cells Most HD cells are found n lateral mammllary nucle (LMN), anteror dorsal thalamc nucleus (ADN), postsubculum (PoS), and retrosplenal cortex (RSC); see Fgure 2. HD cells are also found n the entorhnal cortex (Sargoln et al., 2006) but these cells are downstream from the crcut descrbed here. HD cells are found n both hemspheres and complex crcutry connects the assocated regons. The PoS, whch appears to be the fnal stage contanng pure HD cells, projects nto the entorhnal cortex where some pure HD cells are mxed wth conjunctve cells that extend HD wth spatal propertes that have been proposed to help create postonal maps for gudng spatal navgaton (Fuhs and Touretzky 2006; Burgess et al, 2007; Mhatre et al., 2012; Plly and Grossberg, 2012). Blateral lesons n the LMN or DTN quench HD cells n the ADN (Blar et al., 1998; Bassett et al., 2007) and PoS (Sharp et al., 2006); see Fgure 2. Unlateral LMN lesons do not elmnate the HD cell propertes n ether the ADN or PoS (Blar et al., 1999). Lesons n the ADN dsrupt the HD propertes n the PoS (Goodrdge and Taube, 1997). The PoS also receves nput from vsual regons (Vogt and Mller, 1983; Taube, 1998) and s a good canddate for vson-drven HD preferred drecton resets. Antcpated tme nterval As noted above, HD actvty n the LMN and ADN antcpates the rat s head rotaton (Blar and Sharp, 1995). Ths antcpated tme nterval (ATI) s more promnent n the LMN (70 ms), reduced n the ADN (25 ms), and dsappears n the PoS (Sharp et al., 2001; Taube, 1998, 2007). It has been theorzed that the propertes of the ATI are due to motor and vestbular system nteractons (Taube and Muller, 1997). In the LMN, the recordngs from the psversve hemsphere (the hemsphere towards whch the head s rotatng) showed a larger ATI than the recordngs from the contraversve hemsphere, but n the ADN, the ATI was the same n both hemspheres for psversve and contraversve head turns (Blar et al., 1998).

4 2222 FORTENBERRY ET AL. FIGURE 2. HeadMoVVes model functons [left column] and macrocrcut [rght column]. The HeadMoVVes model requres the multregon crcutry to adaptvely calbrate motor, vestbular, and vsual sgnals nto a relable HD estmate. AHV, angular head velocty; DTN, dorsal tegmental nucleus; LMN, lateral mammllary nucle; ADN, anteror dorsal thalamc nucleus; PoS, postsubculum; RSC, retrosplenal cortex; EC, enthorhnal cortex. Black arrowheads are exctatory connectons, hem-dsks are adaptve connectons, and crcles are nhbtory connectons. The labels n the top rght of each bran regon and letters along the connectons refer to the varables and weghts n the Appendx mathematcal equatons. Rng Attractor Models of Head Drecton Rng attractors have been wdely used to computatonally model HD cells (Blar and Sharp, 1995, 1996; Skaggs et al., 1995; Redsh et al., 1996; Goodrdge and Touretzky, 2000; Boucheny et al., 2005; Song and Wang, 2005). Such rng attractors use a crcular recurrently connected network wth dynamcs whch produce an actvty bump whose poston n the rng represents head drecton. Propertes of ths HD bump are then compared wth the frng characterstcs observed n HD cell recordngs. Integraton of angular head velocty (AHV) sgnals shfts the actvty bump around the rng. The recurrent connectons n the rng attractor can be mplemented usng connectons between the dorsal thalamc nucleus (DTN) and the lateral mammllary nucleus (LMN) (Blar et al., 1998; Song and Wang, 2005) usng nhbtory connectons from DTN to LMN and exctatory connectons from LMN to DTN (Fg. 2). In HD rng attractors, vestbular AHV nputs control the drecton and speed of the actvty bump shft to match the rotaton of the head. Because vestbular AHV nput s a delayed sgnal, t does not explan the antcpatory shft found n the LMN and ADN. Prevous rng attractor models have proposed dfferent mechansms to explan ths antcpatory shft. These mechansms and ther correspondng models are summarzed n the Dscusson secton. As noted above, the HeadMoVVes model proposes that motor outflow sgnals produce antcpaton. The model then needs to clarfy how the bran adaptvely calbrates vsual, vestbular, and motor cues to become dmensonally consstent. To accomplsh ths, the model proposes how multple bran regons contrbute to the calbraton process. Model Overvew The HeadMoVVes model was tested usng a computer smulaton of HD cell responses whle a vrtual artfcal anmal, or anmat, navgates a square envronment n whch vsual landmarks occur n proxmal and dstal locatons. The model HD system was also mplemented n a physcal robot to demonstrate ts capabltes (Fortenberry et al., 2011), but ths applcaton s beyond the scope of ths artcle. In the model, head turns are smulated wth dfferent acceleratons and duratons of rotaton. A Gaussan profle of the

5 ADAPTIVE CALIBRATION OF HEAD DIRECTION 2223 (Fg. 2). The PoS estmates a sngle HD estmate by combnng nformaton across both hemspheres. The vsual nput provded to PoS s used as a fnal error correcton sgnal that anchors the nternally drven HD estmate to vsual landmarks. The vsual, vestbular, and motor outflow sgnals hereby work together to ncrease the relablty of the HD estmate. FIGURE 3. The angular head velocty (AHV) curves and tmng for the HeaDMoVVes model nputs. The motor AHV (dashed lne) curve s the nput to the psversve hemsphere, the vestbular AHV (dash-dotted lne) curve s the nput to the contraversve hemsphere, and the head (sold lne) curve s the velocty of the anmat s head movement. The x-axs shows the angular velocty for one tme step and each smulaton tme step corresponds to 10 ms n duraton. AHV nput through tme s produced wth ampltudes randomly chosen between 2.5 and 2.9, whch equate to deg/s, that determnes the speed of a head rotaton, and a head turn duraton of ms. Fgure 3 shows an example of the nput ampltude set to 2.6 wth duraton of 260 ms. The tme ntervals n the smulatons are computed wth respect to the duratons of head rotatons taken from the data of Sharp (2001). In Fgure 3, the actual head rotaton (sold lne) s plotted aganst tme n 10 ms tme steps and s shown wth the motor AHV nput that s shfted 25 ms earler n tme than the head rotaton (dotted lne) and the vestbular AHV nput that s shfted 25 ms later n tme than the head rotaton (dash-dotted lne) sgnals. Nose s added to the motor and vestbular AHV sgnals to model neuronal and synaptc nose. The motor and vestbular AHV sgnals are sent to the DTN layer of the rng attractor (Fg. 2) and the true head velocty s used to rotate the anmat every 10 ms. Dual hemsphere calbraton The HeadMoVVes model uses a two hemsphere system where one hemsphere receves a motor outflow antcpatory AHV nput and the other receves a vestbular delayed AHV sgnal n response to each head rotaton. The dual hemsphere calbraton allows the system to predct the true head poston durng a turn despte the tme dfference between these two nputs. To accomplsh ths, several bran regons process the nformaton n sequence, each havng a dfferent functon, so that the last HD regon, the PoS, produces the correct tmng (Fg. 2). The DTN and the LMN convert vestbular and motor AHV nto a HD sgnal. The ADN combnes the motor and vestbular HD estmates from the LMNs n both hemspheres Automatc gan control usng habtuatve gates Vestbular and motor nputs may actvate ther target cells wth dfferent gans. The sgnal strengths of correspondng motor and vestbular angular veloctes must match to produce comparable HD drecton shfts for the dual hemsphere nteracton to work correctly. Then the two hemspheres can converge to the same output ndependently of whch nput drves them. How are these nputs adaptvely calbrated to enable successful matchng to occur? A habtuatve gatng mechansm [Appendx Eqs. (A4) and (A5)] s proposed to accomplsh ths. Such a gatng process n each nput pathway tme-averages motor and vestbular nputs over a long tme scale and normalzes the nput sgnals n ts pathway wth ths tme-averaged value, much as happens n retnal photoreceptors (Carpenter and Grossberg, 1981; Grossberg and Hong, 2006), cortcal moton percepton crcuts (Baloch and Grossberg, 1997; Berzhanskaya et al., 2007), and amygdala renforcement learnng crcuts (Grossberg, 1972; Dranas et al., 2008), among other bran systems. As both motor and vestbular nputs become normalzed, the sgnal strengths of vestbular and motor nputs can produce matchng sgnals durng head rotatons wth varable speeds and dstances. Rng attractors n the DTN and LMN Two rng attractors, one n each hemsphere, are used to produce the head drecton representaton from motor and vestbular AHV nputs. The poston of an actvty bump represents the head drecton at any tme. Interactons between the dorsal tegmental nucleus (DTN) and lateral mammllary nucle (LMN) defne each rng attractor (Fgs. 2 and 4). Each hemsphere contans two DTN layers and one LMN layer [Appendx Eqs. (A6) (A12)]. One DTN layer receves vestbular AHV nput and the other receves motor AHV nput. The vestbular AHV nput produces a shft of the actvty bump n the drecton opposte to the hemsphere of the attractor (left shft for the rght hemsphere and rght shft for the left hemsphere). The motor AHV produces a shft of the bump n the drecton that matches the hemsphere of the attractor. In partcular, motor AHV nput to the rght hemsphere causes the rng attractor n the rght hemsphere to shft ts bump to the rght, whereas vestbular AHV nput to the left hemsphere causes the rng attractor n the left hemsphere to also shft ts bump to the rght. Although both rng attractors shft to the rght, the shft due to nput from the rght hemsphere precedes the head rotaton, whereas the shft due to nput from left hemsphere s delayed. Durng a left head rotaton, the stuaton s reversed. The connectons n the rng attractor that produce the actvty bumps and ther shfts n response to motor and vestbular AHV nputs are asymmetrc nhbtory connectons from each

6 2224 FORTENBERRY ET AL. both DTN layers. The DTN nputs equally nhbt the sdes of the LMN bump when the head s motonless. A head rotaton sends ether motor or vestbular angular velocty nput to one DTN layer, ncreasng the asymmetrcal nhbton n one drecton, and shfts the rng n the drecton wth less nhbton. The speed of the shft s proportonal to the dfference of nhbtory nput from the two asymmetrcal connectons. FIGURE 4. Mcrocrcut of the DTN-LMN rng attractor. Two DTN regons and one LMN regon nteract to form a rng attractor. The nhbtory and exctatory connectons between the three layers create a sngle actvty bump that spans multple cells. The two DTNs receve angular head velocty (AHV) nputs. The nhbtory connectons from the two DTNs to the LMN are shfted to opposte drectons. When the two nputs dffer, the nhbtory offset causes the poston of the actvty bump the represents head drecton to shft n the drecton away from that of the larger nhbtory nput. DTN layer to the LMN layer, wth the drecton of the asymmetry opposte for the two DTN layers, and symmetrc exctatory feedback connectons from the LTM to both DTN layers (Fg. 4). The DTN layers also have symmetrc recurrent nhbtory connectons centered on DTN cells that are 1808 away from the source of the connecton to sharpen and stablze the bump. All three layers contnuously receve Posson-dstrbuted nputs that determne a baselne of actvty. The LMN-to-DTN exctatory projecton connects LMN and DTN cells that represent the same head drecton. A bump n the LMN hereby produces a bump n the same poston n Anteror dorsal thalamc nucle The ADN combnes the LMN HD estmates from the two hemspheres to produce a HD estmate that has more precse tmng than one that depends only on one of the motor or vestbular nputs [Fg. 2; Appendx Eqs. (A13) (A14)]. The motordrven psversve LMN estmate s antcpatory, but the vestbular-drven contraversve LMN estmate s delayed for each head rotaton. The ADN actvty adds the two LMN sgnals to create an actvty bump n the poston that s an average of the two LMN estmates. Adjustng the gan parameter F [see Appendx Eqs. (A13) and (A14)] between 0 and 1 adjusts the gan from the contralateral LMN and affects the amount of antcpaton found n the ADN. The sgmodal sgnal functon from the LMN-to-ADN [Appendx Eq. (A15)] postonally sharpens the resultng bump and creates an ADN sgnal that s narrower than each of the LMN sgnals. The comparson of HD estmates n the LMN and ADN of the model wth the expermental data shows correspondence of a sgnal profle and ATI (see Fg. 9 below). Post subculum: Dual hemsphere competton The ADN sends one-to-one topography-preservng connectons to the PoS, as shown n Fgure 5. The PoS makes drect exctatory connectons and ndrect nhbtory connectons va nhbtory nterneurons wth the contralateral PoS. These connectons together form a center-surround network wth a nar- FIGURE 5. Mcrocrcut of the ADN to PoS connectons. These connectons embody the PoS competton. The PoS (P L, P R ) n each hemsphere (L, R) receves a one-to-one exctatory connecton from the correspondng ADN (A L,A R ) cell. Durng a head rotaton, the HD drecton n each hemsphere dffers. The exctatory (W ppex ) and nhbtory (W ppin ) nteractons form a recurrent on-center off-surround network that forces the two PoS HD estmates to compete and merge the two ADN estmates to one poston.

7 row strong on-center and a broad weak off-surround [Appendx Eqs. (A16) and (A17)]. The two recprocally nteractng centersurrounds, one n each hemsphere, drve the HD estmates n the two PoS hemspheres to the same poston. ADAPTIVE CALIBRATION OF HEAD DIRECTION 2225 Retrosplenal cortex: Vsual landmarks Vestbular and motor sgnals alone are not suffcent to estmate the true head poston over tme. Vsual nputs to the PoS (Fg. 2) anchor and stablze the HD cell s preferred drecton to the envronment [Appendx Eqs. (A18) (A20)]. Addng vsual nputs to the model needs to take account of the fact that a sngle vsual cue can match dfferent HD estmates dependng on the poston of the rat n the envronment. For example, f a landmark s postoned n the mddle of the north wall of the envronment and a rat s facng north, the landmark wll appear n the rat s rght hemfeld f the rat s next to the west wall and n the left hemfeld f the rat s next to the east wall. The model learns a mappng between vsual landmarks and the HD estmate n the PoS. To be able to dscrmnate between dfferent postons of the anmal when a landmark s vewed, the model contans object-n-place cells that conjunctvely fuse nformaton about objects and ther postons n the envronment. These cells are organzed n object-place hypercolumns. Each hypercolumn responds to a unque object and each cell wthn the hypercolumn responds to a unque headcentrc spatal poston of the object n the horzontal plane. Such a conjunctve representaton of vsual features and postons s found n the rat bran n areas such as the retrosplenal cortex (RSC) (Vann, 2009). The RSC receves connectons from cortcal area V1 (Wang and Burkhalter, 2007) and the paretal cortex (Taube, 1998). The RSC connects to the ADN and the PoS (Van Groen and Wyss, 1992, 2003; Taube, 2007) and contans HD cells (Chen et al., 1994; Cho and Sharp, 2001). The HeaDMoVVes model does not address the RSCto-ADN connecton because the functonal role of ths connecton s not clear, although the model RSC does project ndrectly to the ADN va the PoS and LMN (Fg. 2). Lesons n the RSC produce less stable HD cells whose drectonal selectvty can drft over tme (Clark et al., 2010). A smlar drft can also occur durng navgaton wth the lghts turned off (Knerm et al., 1998). The model also predcts that RSC lesons wll have a smlar effect on LMN and ADN cells due to the projecton of RSC to these areas va the PoS. Each vsual layer n the model contans 20 cells, where each cell represents 108 of vsual angle. Each landmark actvates two to three RSC cells wth a strength that falls off wth dstance from the landmark poston [Appendx Eq. (A18)]. The combned vsual span of all the cells s 200 degrees (Fg. 6). When a vsual landmark appears n a prescrbed poston and the correspondng cells actvate, the adaptve weghts from the RSC to the PoS [Appendx Eq. (A19)] learn the mappng between the poston of the landmark and the current HD estmate n the PoS. The adaptve weghts are ntalzed to zero to learn a novel envronment. The vson cells start to affect the HD estmate n the PoS as the weghts are establshed through learnng. FIGURE 6. The arrangement of the vsual cues n the envronment. The anmat s confned to the nsde of the box. The rows of crcles on the bottom represent the dfferent vsually actvated object hypercolumns. Each vsual cue actvates a poston n ts hypercolumn when t s n the FOV. Actvaton strength s represented by an nverse grayscale ntensty where whte s zero actvaton. When a vson cell s actve, the weghts ncrease f the PoS cell s actve but decrease f the PoS cell s slent. Over tme the landmarks that are more stable result n a stronger connecton (see Fg. 11 n the Results secton). Dstal landmarks learn to nfluence the PoS more than proxmal landmarks, as n the data (Taube, 1995; Zugaro et al., 2001, 2003, 2004; Yoganarasmha et al., 2006), because the relatve postons of dstal landmarks as the rat moves are more consstent over tme. For a landmark to predct head drecton the relatve poston of the landmark must statstcally have more movement when the rat s head rotates than when the rat mantans the same head drecton but shfts to dfferent postons n the envronment. The statstcs aredetermnedbythedstancetothelandmarkanheszeofthe envronment. As the sze of the envronment ncreases, the landmarks must be more dstal to ensure that the movement of the landmark s a statstcally stronger representaton of head rotatons rather than the rat s poston n the envronment. Ths artcle fxes the sze of the envronment and analyzes the effect the landmark dstance has on the stablty of head drecton sgnal. RESULTS General Setup for Smulatons Wth Vson The smulated envronment s a square enclosure wth vsual landmarks placed ether far from (dstal) or close to (proxmal)

8 2226 FORTENBERRY ET AL. FIGURE 7. The envronment for the proxmal-dstal landmark conflct shft study. The anmat randomly moves wthn the gray square enclosure durng the entre study. After the anmat learns the envronment, the proxmal and dstal landmarks are rotated 908 n dfferent drectons to determne whch landmark the HD estmate follows. A tral s recorded as a dstal shft when the HD estmate shfts between 70 and 1108, a proxmal shft when the HD estmate shfts between 270 and 21108, or a mxed shft when the HD estmate shfts less than 208 or more than 1608 n ether drecton. The gray areas show the regons that consttute a dstal, proxmal, or mxed shft n HD estmate when the anmat s orgnal poston s facng down. the enclosure (Fg. 7). The anmat moves wthn the enclosure, turnng a random drecton every 2 5 s. The anmat moves forward at a constant speed whle rotatng unless t runs nto a wall. When the anmat hts a wall t rotates untl t can run along the wall. Landmarks are postoned at varable dstances n a 3608 crcumference, strategcally placed to reproduce the data of Zugaro (2004) usng the landmark dstrbuton that s descrbed n the next secton. For smplcty, the anmat s eyes n the smulatons were fxed n the head. As a result, landmarks were drectly vewed n head-centrc coordnates, thereby elmnatng the need to transform vsual nputs from retnal coordnates nto head-centrc coordnates that s necessary n case of moveable eyes. Habtuatve gates Vestbular and motor AHV nputs may dffer n strength, for the same movements, f only because the nputs are produced n dfferent regons of the bran. As noted above, the model uses habtuatve gates to transform vestbular and motor AHV nputs nto sgnals that have smlar average gans, and that can thus be matched n a consstent way. To test the performance of the habtuatve gates, three smulatons were performed wth the motor AHV nputs set to 0.7, 1.3, and 1.8 tmes that of the vestbular nputs and constant throughout a tral. The vestbular AHV and head rotaton curves were produced as specfed n Fgure 2. All learnng trals begun wth all the motor and vestbular habtuatve gates set ntally to one, whch s the gate equlbrum value before nputs occur [Eq. (5); Appendx Table A2], and go through a tranng stage and testng stage. Tranng nvolves the anmat movng randomly through the envronment whle constantly changng drecton as n a random foragng task. Learnng occurs untl the habtuatve gate values stablze. Vson was turned off durng the tranng perod n the smulatons presented here to study the drect effect of habtuatve adaptaton wthout the vsual landmark correcton to the PoS. Ths s not requred though, and vsual learnng can go on contnuously even whle habtuaton s occurrng because the habtuaton does not sense the effects of vson, but rather enables vsual learnng to stablze after habtuaton stablzes. After learnng has been establshed, the performance of the model wth habtuatve gates was tested and compared wth the performance of the model wthout habtuatve gates. The habtuatve gates can take as long as 1,000 head rotatons to stablze. The HD estmate n the PoS wll not match the anmat s head rotaton untl the habtuatve gates are stablzed. Fgure 8 descrbes the results of a smulaton of habtuatve gate gan adjustment for a motor AHV gan of 0.7 tmes and 1.8 tmes the vestbular AHV gan n parts (A) and (B), respectvely. Part (A) shows the motor habtuatve gate stablzng near 0.5 to match the vestbular AHV gan. Part (B) shows that the motor habtuatve gate stablzes near 0.2 to decrease the motor AHV gan to match the vestbular AHV gan. In both nstances the vestbular habtuatve gates stablze near Postsubculum and ATI As noted above, HD actvty n the LMN and ADN antcpates the rat s head rotaton (Blar and Sharp, 1995). Ths antcpated tme nterval (ATI) s more promnent n the LMN (70 ms), reduced n the ADN (25 ms), and dsappears n the PoS (Sharp et al., 2001; Taube, 1998, 2007). It has been theorzed that the propertes of the ATI are due to motor and vestbular system nteractons (Taube and Muller, 1997). In the LMN, the recordngs from the psversve hemsphere (the hemsphere towards whch the head s rotatng) showed a larger ATI than the recordngs from the contraversve hemsphere, but n the ADN, the ATI was the same for psversve and contraversve head turns (Blar et al., 1998). In the model, the postsubculum (PoS) competton results n the fnal elmnaton of the ATI and enables the HD estmate to match the anmat s head rotaton. The competton ncludes actvty n both hemspheres and vsual nput (Fg. 2). To test ths effect, a complete head rotaton was plotted from three trals, the frst wth nether the PoS competton nor vson, the second wth the PoS competton but wthout vson, and the thrd wth both PoS competton and vson. The two trals wthout vson took place n a novel envronment wthout vsual landmarks. The tral wth vson used sx dstal landmarks. The vsually actvated adaptve weghts are large enough to nfluence a shft after eght mnutes of learnng, but learnng contnued after that at a rate dependent on how often the vs-

9 ADAPTIVE CALIBRATION OF HEAD DIRECTION 2227 FIGURE 8. The performance of the habtuatve gates. The left column descrbes the vestbular nput (v), ts habtuatve gate (z v ), the gated vestbular nput (H v ) to the DTN layer of the two rng attractors. The rght column descrbes the correspondng quanttes for the motor nput. Although the vestbular and motor nputs are sgnfcantly dfferent, ther gated nputs have a normalzed gan that enables them to be matched. (A) The effect of the habtuatve gates when the motor gan s smaller than the vestbular gan. (B) The effect of the habtuatve gates when the motor gan s larger than the vestbular gan. ual landmarks were vewed. Tranng took place for 30 mn to ensure all the landmarks establshed strong learnng curves. Three plots were produced to show the effect of PoS competton, wth and wthout vson (Fg. 9). Wthout PoS competton, the PoS layers track the ADN layers from the same hemsphere. In partcular, the psversve PoS (dashed lne) tracks the psversve ADN (trangle lne), whle the contraversve PoS (small dotted lne) tracks the contraversve ADN (bg dotted

10 2228 FORTENBERRY ET AL. that lesonng cross-hemspherc nteracton wthn the PoS should lead to dsagreement n HD estmates between the two PoS hemspheres, especally durng rotaton. Ths type of leson wll cause the PoS cells to behave more lke antcpatory ADN cells. When vsual nput s avalable, the two PoS hemspheres (small dotted lne and dashed lne) track the anmat s rotaton and are no longer antcpatory. In Fgures 9B,E, wthout vsual nput from RSC, the two PoS hemspheres are algned to the psversve ADN, but n Fgures 9C,F, wth vsual nput from RSC, the two PoS hemspheres are algned wth the anmat s head poston (sold lne). Thus the model predcts that, wthout vsual nput, the actvty n the two PoS hemspheres wll algn but wll antcpate the head rotaton. HD vsual reset Both vsual nputs va the RSC and vestbular/motor nputs va the ADN nfluence the PoS competton (Fg. 2). Vsual landmarks are anchored to the envronment whle AHV nputs fluctuate wth the dynamc HD estmates n the prevous tme step. As a result, the HD estmate n PoS wll contnually shft towards the vsually guded poston. Ths process occurs wthn ms and often appears on the coarse plots as an mmedate jump to the vsually guded poston. Ths shft s relayed back to the LMN through the PoS-to-LMN feedback connecton (Fg. 2). If two landmarks wthn the vsual feld are ncongruent, the shft wll follow the landmark wth the stronger learned weghts. If only one landmark s present n the vsual feld, the shft wll tend to follow that landmark even f t s a weak landmark. The number of landmarks, the strength of learned weghts, and how long each landmark s n the vsual feld together determne the speed of vsual correcton of the HD estmate. The followng sectons summarze smulatons of the effects of vsual cues under dfferent crcumstances. FIGURE 9. The effect of the PoS competton. The graphs show the tmng of a rotaton for both hemspheres of the ADN and PoS n comparson to the anmat s head rotaton (sold black lne). All lnes to the left of the sold black lne are antcpatory and all lnes to the rght are delayed. The left sde of each mage s the complete head rotaton and the rght s a 50 ms magnfed wndow. [A, D] s the effect wthout PoS competton. [D] the PoS tracks the ADN wthout PoS competton. [B,E] s the effect of PoS competton wthout vsual nput from the RSC. [E] the PoS curves merge towards the same poston between. [C, F] the effect of PoS competton wth vson nput from the RSC. [F] the two PoS curves trackng the poston of the anmat s head rotaton and are no longer antcpatory. lne) (Fgs. 9A,D). When the PoS competton s turned on, the two PoS hemspheres merge towards one another to a poston near the psversve ADN (Fgs. 9B,E). The model predcts HD estmate drft wthout vson There s some evdence of a HD drft when an anmal navgates n the dark (Knerm et al., 1998). The drft s often less than 458 but can drft >908 when the lghts are off for more than 3 mn. The drft wll show varablty and drft back and forth to mantan a drft <458 before drftng to greater dstances. We tested f the model produced a HD drft when vson was not avalable. To test ths, the dfference between the HD estmate and the anmat s head poston was measured every 20 s for 30 mn. Gven that a head rotaton occurs every 2 5 s, the anmat has 5.5 head rotatons every 20 s. Forty trals were run wth 20 vson trals and 20 nonvson trals. The head turn drecton was randomly selected, whch led to rotatons n both the clockwse and counter-clockwse drecton but could have consecutve head rotatons n the same drecton. The trals wth and wthout vson followed the same paradgm as the PoS competton trals descrbed n the prevous secton. The HeadMoVVes model s relable wthout vson and can run for hours wth only modest drfts occurrng. In partcular, out of 20 nonvson trals, 50% drfted less than 108, and 95% drfted less than 208. In one tral the drft dd reach 408 as

11 ADAPTIVE CALIBRATION OF HEAD DIRECTION 2229 FIGURE 10. A smulaton of drft n the HD estmate when vson s avalable (sold lne) and s not avalable (dotted lne). When vson s avalable, the HD estmate sustans the same drecton wthn 88. When vson s not avalable a slow drft can occur. The nonvson drft s rare and s typcally <108. Ths fgure demonstrates the most drastc case from 20 trals. seen n Fgure 10. All vson trals showed drfts that were less than 88 of the true head poston (Fg. 10). Proxmal and dstal vsual landmark dscrmnaton task Zugaro et al. (2004) desgned a task to show that the HD drecton follows dstal rather than proxmal landmarks when these cues rotate n opposte drectons. The Zugaro study used two landmarks placed at dfferent dstances that rotated n opposte drectons. Landmark sze was chosen so that objects appeared the same sze n the rat s feld of vew to avod salency preferences durng learnng. In model smulatons, fve trals were run wth the landmarks placed at dfferent dstances. All landmarks were gven the same sze and salency on the retna for all dstances to avod bases durng learnng. The general layout of the trals can be seen n Fgure 7. All trals set the proxmal and dstal landmarks at the same relatve dstances (1:4) as the Zugaro et al. study but at dfferent absolute dstances to test the dstance effect of the proxmal landmark. The dstances of (proxmal, dstal) landmarks are specfed by Eucldean dstance n pxels from the center of the enclosure. Proxmal and dstal landmarks n the frst two trals were (50, 200) and (100, 400) pxels from the center of the enclosure, wth the proxmal landmark postoned nsde the enclosure. The thrd tral, wth landmarks at (150, 600) pxels from the center of the enclosure, postoned the proxmal landmark just outsde the enclosure. The last tral, wth landmarks at (300, 1,200) pxels from the center of the enclosure, postoned the proxmal landmark sgnfcantly outsde the enclosure. The hypothess s that closer proxmal landmarks have less nfluence on the HD cell s preferred drecton. Keepng the same rato ensures that movng the proxmal landmark does not change the relatve proxmty of the two landmarks. For each tral, the anmat s traned for thrty mnutes, whch s a suffcent duraton to establsh strong vsual learnng, before three measurements were taken of the PoS HD estmate. The left hemsphere PoS HD estmate was arbtrarly used for all trals because each PoS estmate produces the same poston. The HD estmate s measured pror to a landmark shft n order to determne the ntal poston of the HD estmate. Then the proxmal and dstal landmarks were rotated 908 n opposte drectons. After one second a new PoS HD estmate s measured. Ths second measurement determned the amount of shft from the frst recordng. The landmarks were then rotated back to the orgnal poston and, after one more second, a thrd PoS HD estmate was measured. The thrd HD estmate measurement determned the shft accuracy. After ths thrd measurement, the ntally learned weghts were renstated to overcome any partal learnng due to shfts n the landmark postons. Four trangular bns (lght gray regons n Fg. 7) are used to determne the category of shft, one bn for followng the dstal landmark (dstal), one bn for followng the proxmal landmark (proxmal), and another two bns for a combnaton shft of both dstal and proxmal landmarks (mxed). The four bns are 408 wde, 208 n each drecton from the predcted shft n HD drecton. Runs that do not shft to one of the specfed bns were removed from analyss to avod categorzng shfts that cannot be clearly assgned to a sngle bn. The smulatons of the Zugaro et al. (2004) experments are summarzed n Table 1. The fve trals were run wth 40 landmark shfts. Tral 1 wth the shortest dstances (50, 200) had 23 out of 40 landmark shfts removed from analyss because the shfts dd not fall wthn a desgnated bn. Ths was lkely due to nstablty of the proxmal landmarks that are too close to the anmat s postons as t navgates. The fve trals progressvely moved from the HD estmate shft followng the dstal landmark towards followng the two landmarks more equally as the landmarks are moved farther from the center. The frst two trals resulted n a hgher percentage of shfts followng the dstal landmark than the Zugaro et al. (2004) study. These two trals are the only ones that had the proxmal landmarks wthn the anmat s reachable space, wthn the gray box n Fgure 8. The Zugaro et al. (2004) study placed the proxmal landmark outsde the rat s reachable space, whch was smlar to the tral 3 postonng of landmarks. Tral 3 produced the best match to the Zugaro et al. study wth 58% of the shfts followed the dstal landmark, as compared wth 57% n the data; 13% followed the proxmal landmark, as compared to 9% n the data; and 30% followed a mxture between the two, as compared wth 34% n the data. Trals 4 and 5 resulted n the anmat followng the dstal landmark sgnfcantly less than the Zugaro et al. study. Tral 5 resulted n the HD shft followng the proxmal and dstal landmarks equally, whch suggests that the relablty of landmarks s equal for all dstances beyond a threshold. For ths smulaton, the threshold s 300 pxels from the center of the envronment, whch s approxmately twce the radus of the envronment.

12 2230 FORTENBERRY ET AL. TABLE 1. Data and Smulatons for the Proxmal and Dstal Vsual Landmark Dscrmnaton Task Zugaro cm Tral px Tral px Tral px Tral px Tral px Mxed 34% 24% 38% 30% 45% 65% Dstal 57% 71% 60% 58% 38% 18% Proxmal 9% 6% 3% 13% 18% 18% Data and smulatons for the proxmal and dstal vsual landmark dscrmnaton task. The Table shows the percentage of the HD cells that followed the dstal, proxmal or both (mxed) landmarks after they were rotated. The frst column shows the results from the Zugaro et al. (2004). The next fve columns show a parametrc study of the dstal to proxmal landmark relatonshp n the model. Fve dfferent trals were run wth dfferent dstances but the same rato (1:4). Tral 3 shows the closest match to the Zugaro 2004 study, whle Trals 1 and 2 show the effect of movng the landmarks closer and Trals 4 and 5 show the effects of movng the landmarks farther. The relablty of the landmarks s reflected n the learned weghts connectng the vsual landmarks to the PoS. The weghts are larger and more narrowly dstrbuted across PoS neurons for relable landmarks, but weaker and more dffusely dstrbuted across PoS neurons for unrelable landmarks. Fgure 11 shows weghts of sx dfferent landmarks reflectng all the dstances used n the proxmal and dstal dscrmnaton task. The weghts spread across a thrd of the PoS neurons when the landmark was wthn the anmat s obtanable space (sold lne; dstance of 100 pxels). Landmark 2 (dotted lne; dstance of 200 pxels) showed a stronger and narrower tunng curve but sgnfcantly dfferent than the other four. The rest of the landmarks (dstances of 300 1,200 pxels) had smlar whs and ampltudes. The HeadMoVVes model smulates the neurophysology and anatomy of head drecton (HD) cell networks n several bran regons to clarfy how the bran adaptvely calbrates and combnes motor, vsual, and vestbular sgnals to generate relable HD estmates. Each bran regon assocated wth HD cells s predcted to have a specfc functonal role n carryng out ths process. The model adapts the levels of ncomng motor and vestbular angular head velocty sgnals, converts them nto HD sgnals, and uses vsual learnng to anchor HD estmates to famlar vsual landmarks. AHV cells n the DTN have both symmetrcal and asymmetrcal cells. Prevous HD models consder how symmetrcal AHV cells contrbute to HD cell propertes. The HeaDMoVVes model demonstrates how asymmetrcal cells nfluence HD cell propertes. These HD models explan how sgnals from symmetrc AHV cells can shft the HD preferred drecton durng clockwse and counter-clockwse rotatons (Song and Wang, 2005). However, they do not explan why or how antcpaton occurs n the LMN and ADN. The asymmetrc cells convey movement nformaton n only one drecton and can separate nformaton from vestbular and motor sources. The dual hemsphere system allows these two sgnals to be solated and used for learnng, as proposed n the current model. A natural next step n model development would be to combne asymmetrcal and symmetrcal cells to determne whether and how both types of cells can explan a larger range of propertes. A possble role for recprocal PoS-RSC connectons The model assumes that the retrosplenal cortex (RSC) brngs vsual nputs nto the HD system. The only requred connecton for the HeadMoVVes vsual system to work s a connecton from the RSC to the PoS. Ths does not explan DISCUSSION FIGURE 11. Learned weghts from vsual landmarks to the PoS left hemfeld at the sx dstances used n the proxmal and dstal vsual landmark dscrmnaton task. The curves represent the actvty across the PoS neurons when the landmarks are postoned at the center of the vsual feld. Curve heght reflects weght strength. All curves show smlar tunng curves wth the excepton of the landmarks at dstance 100 (sold lne) and 200 (dotted lne) pxels from the center of the envronment. The landmarks at a dstance of 100 pxels s the only one wthn the anmat s envronment (wh and heght of envronment s 240 pxels) and the tunng curve s spread across more PoS neurons wth weaker actvty than all the other landmarks. The landmark at dstance 200 pxels s postoned just outsde the envronment wth a curve that s closer to the other curves but s also a more dspersed and weaker curve.